# Cloud Dataproc API . projects . locations . batches . sparkApplications

## Instance Methods

Obtain high level information corresponding to a single Spark Application.

Obtain environment details for a Spark Application

Obtain data corresponding to a spark job for a Spark Application.

Obtain build data for Native Job

Obtain data corresponding to a particular Native SQL Query for a Spark Application.

Obtain Spark Plan Graph for a Spark Application SQL execution. Limits the number of clusters returned as part of the graph to 10000.

Obtain data corresponding to a particular SQL Query for a Spark Application.

Obtain data corresponding to a spark stage attempt for a Spark Application.

Obtain RDD operation graph for a Spark Application Stage. Limits the number of clusters returned as part of the graph to 10000.

Close httplib2 connections.

Obtain high level information and list of Spark Applications corresponding to a batch

Obtain executor summary with respect to a spark stage attempt.

Retrieves the next page of results.

Obtain data corresponding to executors for a Spark Application.

Retrieves the next page of results.

Obtain list of spark jobs corresponding to a Spark Application.

Retrieves the next page of results.

Obtain data corresponding to Native SQL Queries for a Spark Application.

Retrieves the next page of results.

Obtain data corresponding to SQL Queries for a Spark Application.

Retrieves the next page of results.

Obtain data corresponding to tasks for a spark stage attempt for a Spark Application.

Retrieves the next page of results.

Obtain data corresponding to a spark stage attempts for a Spark Application.

Retrieves the next page of results.

Obtain data corresponding to stages for a Spark Application.

Retrieves the next page of results.

Retrieves the next page of results.

Obtain summary of Executor Summary for a Spark Application

Obtain summary of Jobs for a Spark Application

Obtain summary of Tasks for a Spark Application Stage Attempt

Obtain summary of Stages for a Spark Application

Write wrapper objects from dataplane to spanner

### Method Details

`access(name, parent=None, x__xgafv=None)`

    Obtain high level information corresponding to a single Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # A summary of Spark Application
      "application": { # High level information corresponding to an application. # Output only. High level information corresponding to an application.
        "applicationContextIngestionStatus": "A String",
        "applicationId": "A String",
        "attempts": [
          { # Specific attempt of an application.
            "appSparkVersion": "A String",
            "attemptId": "A String",
            "completed": True or False,
            "durationMillis": "A String",
            "endTime": "A String",
            "lastUpdated": "A String",
            "sparkUser": "A String",
            "startTime": "A String",
          },
        ],
        "coresGranted": 42,
        "coresPerExecutor": 42,
        "maxCores": 42,
        "memoryPerExecutorMb": 42,
        "name": "A String",
        "quantileDataStatus": "A String",
      },
    }

`accessEnvironmentInfo(name, parent=None, x__xgafv=None)`

    Obtain environment details for a Spark Application

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Environment details of a Saprk Application.
      "applicationEnvironmentInfo": { # Details about the Environment that the application is running in. # Details about the Environment that the application is running in.
        "classpathEntries": {
          "a_key": "A String",
        },
        "hadoopProperties": {
          "a_key": "A String",
        },
        "metricsProperties": {
          "a_key": "A String",
        },
        "resourceProfiles": [
          { # Resource profile that contains information about all the resources required by executors and tasks.
            "executorResources": {
              "a_key": { # Resources used per executor used by the application.
                "amount": "A String",
                "discoveryScript": "A String",
                "resourceName": "A String",
                "vendor": "A String",
              },
            },
            "resourceProfileId": 42,
            "taskResources": {
              "a_key": { # Resources used per task created by the application.
                "amount": 3.14,
                "resourceName": "A String",
              },
            },
          },
        ],
        "runtime": {
          "javaHome": "A String",
          "javaVersion": "A String",
          "scalaVersion": "A String",
        },
        "sparkProperties": {
          "a_key": "A String",
        },
        "systemProperties": {
          "a_key": "A String",
        },
      },
    }

`accessJob(name, jobId=None, parent=None, x__xgafv=None)`

    Obtain data corresponding to a spark job for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      jobId: string, Required. Job ID to fetch data for.
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Details of a particular job associated with Spark Application
      "jobData": { # Data corresponding to a spark job. # Output only. Data corresponding to a spark job.
        "completionTime": "A String",
        "description": "A String",
        "jobGroup": "A String",
        "jobId": "A String",
        "killTasksSummary": {
          "a_key": 42,
        },
        "name": "A String",
        "numActiveStages": 42,
        "numActiveTasks": 42,
        "numCompletedIndices": 42,
        "numCompletedStages": 42,
        "numCompletedTasks": 42,
        "numFailedStages": 42,
        "numFailedTasks": 42,
        "numKilledTasks": 42,
        "numSkippedStages": 42,
        "numSkippedTasks": 42,
        "numTasks": 42,
        "skippedStages": [
          42,
        ],
        "sqlExecutionId": "A String",
        "stageIds": [
          "A String",
        ],
        "status": "A String",
        "submissionTime": "A String",
      },
    }

`accessNativeBuildInfo(name, parent=None, x__xgafv=None)`

    Obtain build data for Native Job

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Details of Native Build Info for a Spark Application
      "buildInfo": { # Native Build Info Data
        "buildClass": "A String", # Optional. Build class of Native.
        "buildInfo": [ # Optional. Build related details.
          { # Native Build Info
            "buildKey": "A String", # Optional. Build key.
            "buildValue": "A String", # Optional. Build value.
          },
        ],
      },
    }

`accessNativeSqlQuery(name, executionId=None, parent=None, x__xgafv=None)`

    Obtain data corresponding to a particular Native SQL Query for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      executionId: string, Required. Execution ID
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Details of a query for a Spark Application
      "executionData": { # Native SQL Execution Data # Native SQL Execution Data
        "description": "A String", # Optional. Description of the execution.
        "executionId": "A String", # Required. Execution ID of the Native SQL Execution.
        "fallbackDescription": "A String", # Optional. Description of the fallback.
        "fallbackNodeToReason": [ # Optional. Fallback node to reason.
          { # Native SQL Execution Data
            "fallbackNode": "A String", # Optional. Fallback node information.
            "fallbackReason": "A String", # Optional. Fallback to Spark reason.
          },
        ],
        "numFallbackNodes": 42, # Optional. Number of nodes fallen back to Spark.
        "numNativeNodes": 42, # Optional. Number of nodes in Native.
      },
    }

`accessSqlPlan(name, executionId=None, parent=None, x__xgafv=None)`

    Obtain Spark Plan Graph for a Spark Application SQL execution. Limits the number of clusters returned as part of the graph to 10000.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      executionId: string, Required. Execution ID
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # SparkPlanGraph for a Spark Application execution limited to maximum 10000 clusters.
      "sparkPlanGraph": { # A graph used for storing information of an executionPlan of DataFrame. # SparkPlanGraph for a Spark Application execution.
        "edges": [
          { # Represents a directed edge in the spark plan tree from child to parent.
            "fromId": "A String",
            "toId": "A String",
          },
        ],
        "executionId": "A String",
        "nodes": [
          { # Wrapper user to represent either a node or a cluster.
            "cluster": { # Represents a tree of spark plan.
              "desc": "A String",
              "metrics": [
                { # Metrics related to SQL execution.
                  "accumulatorId": "A String",
                  "metricType": "A String",
                  "name": "A String",
                },
              ],
              "name": "A String",
              "nodes": [
                # Object with schema name: SparkPlanGraphNodeWrapper
              ],
              "sparkPlanGraphClusterId": "A String",
            },
            "node": { # Represents a node in the spark plan tree.
              "desc": "A String",
              "metrics": [
                { # Metrics related to SQL execution.
                  "accumulatorId": "A String",
                  "metricType": "A String",
                  "name": "A String",
                },
              ],
              "name": "A String",
              "sparkPlanGraphNodeId": "A String",
            },
          },
        ],
      },
    }

`accessSqlQuery(name, details=None, executionId=None, parent=None, planDescription=None, x__xgafv=None)`

    Obtain data corresponding to a particular SQL Query for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      details: boolean, Optional. Lists/ hides details of Spark plan nodes. True is set to list and false to hide.
      executionId: string, Required. Execution ID
      parent: string, Required. Parent (Batch) resource reference.
      planDescription: boolean, Optional. Enables/ disables physical plan description on demand
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Details of a query for a Spark Application
      "executionData": { # SQL Execution Data # SQL Execution Data
        "completionTime": "A String",
        "description": "A String",
        "details": "A String",
        "errorMessage": "A String",
        "executionId": "A String",
        "jobs": {
          "a_key": "A String",
        },
        "metricValues": {
          "a_key": "A String",
        },
        "metricValuesIsNull": True or False,
        "metrics": [
          { # Metrics related to SQL execution.
            "accumulatorId": "A String",
            "metricType": "A String",
            "name": "A String",
          },
        ],
        "modifiedConfigs": {
          "a_key": "A String",
        },
        "physicalPlanDescription": "A String",
        "rootExecutionId": "A String",
        "stages": [
          "A String",
        ],
        "submissionTime": "A String",
      },
    }

`accessStageAttempt(name, parent=None, stageAttemptId=None, stageId=None, summaryMetricsMask=None, x__xgafv=None)`

    Obtain data corresponding to a spark stage attempt for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      stageAttemptId: integer, Required. Stage Attempt ID
      stageId: string, Required. Stage ID
      summaryMetricsMask: string, Optional. The list of summary metrics fields to include. Empty list will default to skip all summary metrics fields. Example, if the response should include TaskQuantileMetrics, the request should have task_quantile_metrics in summary_metrics_mask field
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Stage Attempt for a Stage of a Spark Application
      "stageData": { # Data corresponding to a stage. # Output only. Data corresponding to a stage.
        "accumulatorUpdates": [
          {
            "accumullableInfoId": "A String",
            "name": "A String",
            "update": "A String",
            "value": "A String",
          },
        ],
        "completionTime": "A String",
        "description": "A String",
        "details": "A String",
        "executorMetricsDistributions": {
          "diskBytesSpilled": [
            3.14,
          ],
          "failedTasks": [
            3.14,
          ],
          "inputBytes": [
            3.14,
          ],
          "inputRecords": [
            3.14,
          ],
          "killedTasks": [
            3.14,
          ],
          "memoryBytesSpilled": [
            3.14,
          ],
          "outputBytes": [
            3.14,
          ],
          "outputRecords": [
            3.14,
          ],
          "peakMemoryMetrics": {
            "executorMetrics": [
              {
                "metrics": {
                  "a_key": "A String",
                },
              },
            ],
            "quantiles": [
              3.14,
            ],
          },
          "quantiles": [
            3.14,
          ],
          "shuffleRead": [
            3.14,
          ],
          "shuffleReadRecords": [
            3.14,
          ],
          "shuffleWrite": [
            3.14,
          ],
          "shuffleWriteRecords": [
            3.14,
          ],
          "succeededTasks": [
            3.14,
          ],
          "taskTimeMillis": [
            3.14,
          ],
        },
        "executorSummary": {
          "a_key": { # Executor resources consumed by a stage.
            "diskBytesSpilled": "A String",
            "executorId": "A String",
            "failedTasks": 42,
            "inputBytes": "A String",
            "inputRecords": "A String",
            "isExcludedForStage": True or False,
            "killedTasks": 42,
            "memoryBytesSpilled": "A String",
            "outputBytes": "A String",
            "outputRecords": "A String",
            "peakMemoryMetrics": {
              "metrics": {
                "a_key": "A String",
              },
            },
            "shuffleRead": "A String",
            "shuffleReadRecords": "A String",
            "shuffleWrite": "A String",
            "shuffleWriteRecords": "A String",
            "stageAttemptId": 42,
            "stageId": "A String",
            "succeededTasks": 42,
            "taskTimeMillis": "A String",
          },
        },
        "failureReason": "A String",
        "firstTaskLaunchedTime": "A String",
        "isShufflePushEnabled": True or False,
        "jobIds": [
          "A String",
        ],
        "killedTasksSummary": {
          "a_key": 42,
        },
        "locality": {
          "a_key": "A String",
        },
        "name": "A String",
        "numActiveTasks": 42,
        "numCompleteTasks": 42,
        "numCompletedIndices": 42,
        "numFailedTasks": 42,
        "numKilledTasks": 42,
        "numTasks": 42,
        "parentStageIds": [
          "A String",
        ],
        "peakExecutorMetrics": {
          "metrics": {
            "a_key": "A String",
          },
        },
        "rddIds": [
          "A String",
        ],
        "resourceProfileId": 42,
        "schedulingPool": "A String",
        "shuffleMergersCount": 42,
        "speculationSummary": { # Details of the speculation task when speculative execution is enabled.
          "numActiveTasks": 42,
          "numCompletedTasks": 42,
          "numFailedTasks": 42,
          "numKilledTasks": 42,
          "numTasks": 42,
          "stageAttemptId": 42,
          "stageId": "A String",
        },
        "stageAttemptId": 42,
        "stageId": "A String",
        "stageMetrics": { # Stage Level Aggregated Metrics
          "diskBytesSpilled": "A String",
          "executorCpuTimeNanos": "A String",
          "executorDeserializeCpuTimeNanos": "A String",
          "executorDeserializeTimeMillis": "A String",
          "executorRunTimeMillis": "A String",
          "jvmGcTimeMillis": "A String",
          "memoryBytesSpilled": "A String",
          "peakExecutionMemoryBytes": "A String",
          "resultSerializationTimeMillis": "A String",
          "resultSize": "A String",
          "stageInputMetrics": { # Metrics about the input read by the stage.
            "bytesRead": "A String",
            "recordsRead": "A String",
          },
          "stageOutputMetrics": { # Metrics about the output written by the stage.
            "bytesWritten": "A String",
            "recordsWritten": "A String",
          },
          "stageShuffleReadMetrics": { # Shuffle data read for the stage.
            "bytesRead": "A String",
            "fetchWaitTimeMillis": "A String",
            "localBlocksFetched": "A String",
            "localBytesRead": "A String",
            "recordsRead": "A String",
            "remoteBlocksFetched": "A String",
            "remoteBytesRead": "A String",
            "remoteBytesReadToDisk": "A String",
            "remoteReqsDuration": "A String",
            "stageShufflePushReadMetrics": {
              "corruptMergedBlockChunks": "A String",
              "localMergedBlocksFetched": "A String",
              "localMergedBytesRead": "A String",
              "localMergedChunksFetched": "A String",
              "mergedFetchFallbackCount": "A String",
              "remoteMergedBlocksFetched": "A String",
              "remoteMergedBytesRead": "A String",
              "remoteMergedChunksFetched": "A String",
              "remoteMergedReqsDuration": "A String",
            },
          },
          "stageShuffleWriteMetrics": { # Shuffle data written for the stage.
            "bytesWritten": "A String",
            "recordsWritten": "A String",
            "writeTimeNanos": "A String",
          },
        },
        "status": "A String",
        "submissionTime": "A String",
        "taskQuantileMetrics": { # Summary metrics fields. These are included in response only if present in summary_metrics_mask field in request
          "diskBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "durationMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "executorCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "executorDeserializeCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "executorDeserializeTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "executorRunTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "gettingResultTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "inputMetrics": {
            "bytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "recordsRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
          },
          "jvmGcTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "memoryBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "outputMetrics": {
            "bytesWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "recordsWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
          },
          "peakExecutionMemoryBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "resultSerializationTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "resultSize": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "schedulerDelayMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
            "count": "A String",
            "maximum": "A String",
            "minimum": "A String",
            "percentile25": "A String",
            "percentile50": "A String",
            "percentile75": "A String",
            "sum": "A String",
          },
          "shuffleReadMetrics": {
            "fetchWaitTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "localBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "readBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "readRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "remoteBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "remoteBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "remoteBytesReadToDisk": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "remoteReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "shufflePushReadMetrics": {
              "corruptMergedBlockChunks": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "localMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "localMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "localMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "mergedFetchFallbackCount": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteMergedReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "totalBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
          },
          "shuffleWriteMetrics": {
            "writeBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "writeRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "writeTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
          },
        },
        "tasks": {
          "a_key": { # Data corresponding to tasks created by spark.
            "accumulatorUpdates": [
              {
                "accumullableInfoId": "A String",
                "name": "A String",
                "update": "A String",
                "value": "A String",
              },
            ],
            "attempt": 42,
            "durationMillis": "A String",
            "errorMessage": "A String",
            "executorId": "A String",
            "executorLogs": {
              "a_key": "A String",
            },
            "gettingResultTimeMillis": "A String",
            "hasMetrics": True or False,
            "host": "A String",
            "index": 42,
            "launchTime": "A String",
            "partitionId": 42,
            "resultFetchStart": "A String",
            "schedulerDelayMillis": "A String",
            "speculative": True or False,
            "stageAttemptId": 42,
            "stageId": "A String",
            "status": "A String",
            "taskId": "A String",
            "taskLocality": "A String",
            "taskMetrics": { # Executor Task Metrics
              "diskBytesSpilled": "A String",
              "executorCpuTimeNanos": "A String",
              "executorDeserializeCpuTimeNanos": "A String",
              "executorDeserializeTimeMillis": "A String",
              "executorRunTimeMillis": "A String",
              "inputMetrics": { # Metrics about the input data read by the task.
                "bytesRead": "A String",
                "recordsRead": "A String",
              },
              "jvmGcTimeMillis": "A String",
              "memoryBytesSpilled": "A String",
              "outputMetrics": { # Metrics about the data written by the task.
                "bytesWritten": "A String",
                "recordsWritten": "A String",
              },
              "peakExecutionMemoryBytes": "A String",
              "resultSerializationTimeMillis": "A String",
              "resultSize": "A String",
              "shuffleReadMetrics": { # Shuffle data read by the task.
                "fetchWaitTimeMillis": "A String",
                "localBlocksFetched": "A String",
                "localBytesRead": "A String",
                "recordsRead": "A String",
                "remoteBlocksFetched": "A String",
                "remoteBytesRead": "A String",
                "remoteBytesReadToDisk": "A String",
                "remoteReqsDuration": "A String",
                "shufflePushReadMetrics": {
                  "corruptMergedBlockChunks": "A String",
                  "localMergedBlocksFetched": "A String",
                  "localMergedBytesRead": "A String",
                  "localMergedChunksFetched": "A String",
                  "mergedFetchFallbackCount": "A String",
                  "remoteMergedBlocksFetched": "A String",
                  "remoteMergedBytesRead": "A String",
                  "remoteMergedChunksFetched": "A String",
                  "remoteMergedReqsDuration": "A String",
                },
              },
              "shuffleWriteMetrics": { # Shuffle data written by task.
                "bytesWritten": "A String",
                "recordsWritten": "A String",
                "writeTimeNanos": "A String",
              },
            },
          },
        },
      },
    }

`accessStageRddGraph(name, parent=None, stageId=None, x__xgafv=None)`

    Obtain RDD operation graph for a Spark Application Stage. Limits the number of clusters returned as part of the graph to 10000.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      stageId: string, Required. Stage ID
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # RDD operation graph for a Spark Application Stage limited to maximum 10000 clusters.
      "rddOperationGraph": { # Graph representing RDD dependencies. Consists of edges and a root cluster. # RDD operation graph for a Spark Application Stage.
        "edges": [
          { # A directed edge representing dependency between two RDDs.
            "fromId": 42,
            "toId": 42,
          },
        ],
        "incomingEdges": [
          { # A directed edge representing dependency between two RDDs.
            "fromId": 42,
            "toId": 42,
          },
        ],
        "outgoingEdges": [
          { # A directed edge representing dependency between two RDDs.
            "fromId": 42,
            "toId": 42,
          },
        ],
        "rootCluster": { # A grouping of nodes representing higher level constructs (stage, job etc.).
          "childClusters": [
            # Object with schema name: RddOperationCluster
          ],
          "childNodes": [
            { # A node in the RDD operation graph. Corresponds to a single RDD.
              "barrier": True or False,
              "cached": True or False,
              "callsite": "A String",
              "name": "A String",
              "nodeId": 42,
              "outputDeterministicLevel": "A String",
            },
          ],
          "name": "A String",
          "rddClusterId": "A String",
        },
        "stageId": "A String",
      },
    }

`close()`

    Close httplib2 connections.

`search(parent, applicationStatus=None, maxEndTime=None, maxTime=None, minEndTime=None, minTime=None, pageSize=None, pageToken=None, x__xgafv=None)`

    Obtain high level information and list of Spark Applications corresponding to a batch

    Args:
      parent: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID" (required)
      applicationStatus: string, Optional. Search only applications in the chosen state.
        Allowed values
          APPLICATION_STATUS_UNSPECIFIED -
          APPLICATION_STATUS_RUNNING -
          APPLICATION_STATUS_COMPLETED -
      maxEndTime: string, Optional. Latest end timestamp to list.
      maxTime: string, Optional. Latest start timestamp to list.
      minEndTime: string, Optional. Earliest end timestamp to list.
      minTime: string, Optional. Earliest start timestamp to list.
      pageSize: integer, Optional. Maximum number of applications to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous SearchSparkApplications call. Provide this token to retrieve the subsequent page.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # A list of summary of Spark Applications
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationsRequest.
      "sparkApplications": [ # Output only. High level information corresponding to an application.
        { # A summary of Spark Application
          "application": { # High level information corresponding to an application. # Output only. High level information corresponding to an application.
            "applicationContextIngestionStatus": "A String",
            "applicationId": "A String",
            "attempts": [
              { # Specific attempt of an application.
                "appSparkVersion": "A String",
                "attemptId": "A String",
                "completed": True or False,
                "durationMillis": "A String",
                "endTime": "A String",
                "lastUpdated": "A String",
                "sparkUser": "A String",
                "startTime": "A String",
              },
            ],
            "coresGranted": 42,
            "coresPerExecutor": 42,
            "maxCores": 42,
            "memoryPerExecutorMb": 42,
            "name": "A String",
            "quantileDataStatus": "A String",
          },
          "name": "A String", # Identifier. Name of the spark application
        },
      ],
    }

`searchExecutorStageSummary(name, pageSize=None, pageToken=None, parent=None, stageAttemptId=None, stageId=None, x__xgafv=None)`

    Obtain executor summary with respect to a spark stage attempt.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      pageSize: integer, Optional. Maximum number of executors to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous AccessSparkApplicationExecutorsList call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      stageAttemptId: integer, Required. Stage Attempt ID
      stageId: string, Required. Stage ID
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # List of Executors associated with a Spark Application Stage.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationExecutorsListRequest.
      "sparkApplicationStageExecutors": [ # Details about executors used by the application stage.
        { # Executor resources consumed by a stage.
          "diskBytesSpilled": "A String",
          "executorId": "A String",
          "failedTasks": 42,
          "inputBytes": "A String",
          "inputRecords": "A String",
          "isExcludedForStage": True or False,
          "killedTasks": 42,
          "memoryBytesSpilled": "A String",
          "outputBytes": "A String",
          "outputRecords": "A String",
          "peakMemoryMetrics": {
            "metrics": {
              "a_key": "A String",
            },
          },
          "shuffleRead": "A String",
          "shuffleReadRecords": "A String",
          "shuffleWrite": "A String",
          "shuffleWriteRecords": "A String",
          "stageAttemptId": 42,
          "stageId": "A String",
          "succeededTasks": 42,
          "taskTimeMillis": "A String",
        },
      ],
    }

`searchExecutorStageSummary_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchExecutors(name, executorStatus=None, pageSize=None, pageToken=None, parent=None, x__xgafv=None)`

    Obtain data corresponding to executors for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      executorStatus: string, Optional. Filter to select whether active/ dead or all executors should be selected.
        Allowed values
          EXECUTOR_STATUS_UNSPECIFIED -
          EXECUTOR_STATUS_ACTIVE -
          EXECUTOR_STATUS_DEAD -
      pageSize: integer, Optional. Maximum number of executors to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous AccessSparkApplicationExecutorsList call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # List of Executors associated with a Spark Application.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationExecutorsListRequest.
      "sparkApplicationExecutors": [ # Details about executors used by the application.
        { # Details about executors used by the application.
          "activeTasks": 42,
          "addTime": "A String",
          "attributes": {
            "a_key": "A String",
          },
          "completedTasks": 42,
          "diskUsed": "A String",
          "excludedInStages": [
            "A String",
          ],
          "executorId": "A String",
          "executorLogs": {
            "a_key": "A String",
          },
          "failedTasks": 42,
          "hostPort": "A String",
          "isActive": True or False,
          "isExcluded": True or False,
          "maxMemory": "A String",
          "maxTasks": 42,
          "memoryMetrics": {
            "totalOffHeapStorageMemory": "A String",
            "totalOnHeapStorageMemory": "A String",
            "usedOffHeapStorageMemory": "A String",
            "usedOnHeapStorageMemory": "A String",
          },
          "memoryUsed": "A String",
          "peakMemoryMetrics": {
            "metrics": {
              "a_key": "A String",
            },
          },
          "rddBlocks": 42,
          "removeReason": "A String",
          "removeTime": "A String",
          "resourceProfileId": 42,
          "resources": {
            "a_key": {
              "addresses": [
                "A String",
              ],
              "name": "A String",
            },
          },
          "totalCores": 42,
          "totalDurationMillis": "A String",
          "totalGcTimeMillis": "A String",
          "totalInputBytes": "A String",
          "totalShuffleRead": "A String",
          "totalShuffleWrite": "A String",
          "totalTasks": 42,
        },
      ],
    }

`searchExecutors_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchJobs(name, jobStatus=None, pageSize=None, pageToken=None, parent=None, x__xgafv=None)`

    Obtain list of spark jobs corresponding to a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      jobStatus: string, Optional. List only jobs in the specific state.
        Allowed values
          JOB_EXECUTION_STATUS_UNSPECIFIED -
          JOB_EXECUTION_STATUS_RUNNING -
          JOB_EXECUTION_STATUS_SUCCEEDED -
          JOB_EXECUTION_STATUS_FAILED -
          JOB_EXECUTION_STATUS_UNKNOWN -
      pageSize: integer, Optional. Maximum number of jobs to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous SearchSparkApplicationJobs call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # A list of Jobs associated with a Spark Application.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationJobsRequest.
      "sparkApplicationJobs": [ # Output only. Data corresponding to a spark job.
        { # Data corresponding to a spark job.
          "completionTime": "A String",
          "description": "A String",
          "jobGroup": "A String",
          "jobId": "A String",
          "killTasksSummary": {
            "a_key": 42,
          },
          "name": "A String",
          "numActiveStages": 42,
          "numActiveTasks": 42,
          "numCompletedIndices": 42,
          "numCompletedStages": 42,
          "numCompletedTasks": 42,
          "numFailedStages": 42,
          "numFailedTasks": 42,
          "numKilledTasks": 42,
          "numSkippedStages": 42,
          "numSkippedTasks": 42,
          "numTasks": 42,
          "skippedStages": [
            42,
          ],
          "sqlExecutionId": "A String",
          "stageIds": [
            "A String",
          ],
          "status": "A String",
          "submissionTime": "A String",
        },
      ],
    }

`searchJobs_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchNativeSqlQueries(name, pageSize=None, pageToken=None, parent=None, x__xgafv=None)`

    Obtain data corresponding to Native SQL Queries for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      pageSize: integer, Optional. Maximum number of queries to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous SearchSparkApplicationNativeSqlQueries call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # List of all Native SQL queries details for a Spark Application.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationNativeSqlQueriesRequest.
      "sparkApplicationNativeSqlQueries": [ # Output only. Native SQL Execution Data
        { # Native SQL Execution Data
          "description": "A String", # Optional. Description of the execution.
          "executionId": "A String", # Required. Execution ID of the Native SQL Execution.
          "fallbackDescription": "A String", # Optional. Description of the fallback.
          "fallbackNodeToReason": [ # Optional. Fallback node to reason.
            { # Native SQL Execution Data
              "fallbackNode": "A String", # Optional. Fallback node information.
              "fallbackReason": "A String", # Optional. Fallback to Spark reason.
            },
          ],
          "numFallbackNodes": 42, # Optional. Number of nodes fallen back to Spark.
          "numNativeNodes": 42, # Optional. Number of nodes in Native.
        },
      ],
    }

`searchNativeSqlQueries_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchSqlQueries(name, details=None, pageSize=None, pageToken=None, parent=None, planDescription=None, x__xgafv=None)`

    Obtain data corresponding to SQL Queries for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      details: boolean, Optional. Lists/ hides details of Spark plan nodes. True is set to list and false to hide.
      pageSize: integer, Optional. Maximum number of queries to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous SearchSparkApplicationSqlQueries call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      planDescription: boolean, Optional. Enables/ disables physical plan description on demand
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # List of all queries for a Spark Application.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationSqlQueriesRequest.
      "sparkApplicationSqlQueries": [ # Output only. SQL Execution Data
        { # SQL Execution Data
          "completionTime": "A String",
          "description": "A String",
          "details": "A String",
          "errorMessage": "A String",
          "executionId": "A String",
          "jobs": {
            "a_key": "A String",
          },
          "metricValues": {
            "a_key": "A String",
          },
          "metricValuesIsNull": True or False,
          "metrics": [
            { # Metrics related to SQL execution.
              "accumulatorId": "A String",
              "metricType": "A String",
              "name": "A String",
            },
          ],
          "modifiedConfigs": {
            "a_key": "A String",
          },
          "physicalPlanDescription": "A String",
          "rootExecutionId": "A String",
          "stages": [
            "A String",
          ],
          "submissionTime": "A String",
        },
      ],
    }

`searchSqlQueries_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchStageAttemptTasks(name, pageSize=None, pageToken=None, parent=None, sortRuntime=None, stageAttemptId=None, stageId=None, taskStatus=None, x__xgafv=None)`

    Obtain data corresponding to tasks for a spark stage attempt for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      pageSize: integer, Optional. Maximum number of tasks to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous ListSparkApplicationStageAttemptTasks call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      sortRuntime: boolean, Optional. Sort the tasks by runtime.
      stageAttemptId: integer, Optional. Stage Attempt ID
      stageId: string, Optional. Stage ID
      taskStatus: string, Optional. List only tasks in the state.
        Allowed values
          TASK_STATUS_UNSPECIFIED -
          TASK_STATUS_RUNNING -
          TASK_STATUS_SUCCESS -
          TASK_STATUS_FAILED -
          TASK_STATUS_KILLED -
          TASK_STATUS_PENDING -
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # List of tasks for a stage of a Spark Application
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent ListSparkApplicationStageAttemptTasksRequest.
      "sparkApplicationStageAttemptTasks": [ # Output only. Data corresponding to tasks created by spark.
        { # Data corresponding to tasks created by spark.
          "accumulatorUpdates": [
            {
              "accumullableInfoId": "A String",
              "name": "A String",
              "update": "A String",
              "value": "A String",
            },
          ],
          "attempt": 42,
          "durationMillis": "A String",
          "errorMessage": "A String",
          "executorId": "A String",
          "executorLogs": {
            "a_key": "A String",
          },
          "gettingResultTimeMillis": "A String",
          "hasMetrics": True or False,
          "host": "A String",
          "index": 42,
          "launchTime": "A String",
          "partitionId": 42,
          "resultFetchStart": "A String",
          "schedulerDelayMillis": "A String",
          "speculative": True or False,
          "stageAttemptId": 42,
          "stageId": "A String",
          "status": "A String",
          "taskId": "A String",
          "taskLocality": "A String",
          "taskMetrics": { # Executor Task Metrics
            "diskBytesSpilled": "A String",
            "executorCpuTimeNanos": "A String",
            "executorDeserializeCpuTimeNanos": "A String",
            "executorDeserializeTimeMillis": "A String",
            "executorRunTimeMillis": "A String",
            "inputMetrics": { # Metrics about the input data read by the task.
              "bytesRead": "A String",
              "recordsRead": "A String",
            },
            "jvmGcTimeMillis": "A String",
            "memoryBytesSpilled": "A String",
            "outputMetrics": { # Metrics about the data written by the task.
              "bytesWritten": "A String",
              "recordsWritten": "A String",
            },
            "peakExecutionMemoryBytes": "A String",
            "resultSerializationTimeMillis": "A String",
            "resultSize": "A String",
            "shuffleReadMetrics": { # Shuffle data read by the task.
              "fetchWaitTimeMillis": "A String",
              "localBlocksFetched": "A String",
              "localBytesRead": "A String",
              "recordsRead": "A String",
              "remoteBlocksFetched": "A String",
              "remoteBytesRead": "A String",
              "remoteBytesReadToDisk": "A String",
              "remoteReqsDuration": "A String",
              "shufflePushReadMetrics": {
                "corruptMergedBlockChunks": "A String",
                "localMergedBlocksFetched": "A String",
                "localMergedBytesRead": "A String",
                "localMergedChunksFetched": "A String",
                "mergedFetchFallbackCount": "A String",
                "remoteMergedBlocksFetched": "A String",
                "remoteMergedBytesRead": "A String",
                "remoteMergedChunksFetched": "A String",
                "remoteMergedReqsDuration": "A String",
              },
            },
            "shuffleWriteMetrics": { # Shuffle data written by task.
              "bytesWritten": "A String",
              "recordsWritten": "A String",
              "writeTimeNanos": "A String",
            },
          },
        },
      ],
    }

`searchStageAttemptTasks_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchStageAttempts(name, pageSize=None, pageToken=None, parent=None, stageId=None, summaryMetricsMask=None, x__xgafv=None)`

    Obtain data corresponding to a spark stage attempts for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      pageSize: integer, Optional. Maximum number of stage attempts (paging based on stage_attempt_id) to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous SearchSparkApplicationStageAttempts call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      stageId: string, Required. Stage ID for which attempts are to be fetched
      summaryMetricsMask: string, Optional. The list of summary metrics fields to include. Empty list will default to skip all summary metrics fields. Example, if the response should include TaskQuantileMetrics, the request should have task_quantile_metrics in summary_metrics_mask field
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # A list of Stage Attempts for a Stage of a Spark Application.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent ListSparkApplicationStageAttemptsRequest.
      "sparkApplicationStageAttempts": [ # Output only. Data corresponding to a stage attempts
        { # Data corresponding to a stage.
          "accumulatorUpdates": [
            {
              "accumullableInfoId": "A String",
              "name": "A String",
              "update": "A String",
              "value": "A String",
            },
          ],
          "completionTime": "A String",
          "description": "A String",
          "details": "A String",
          "executorMetricsDistributions": {
            "diskBytesSpilled": [
              3.14,
            ],
            "failedTasks": [
              3.14,
            ],
            "inputBytes": [
              3.14,
            ],
            "inputRecords": [
              3.14,
            ],
            "killedTasks": [
              3.14,
            ],
            "memoryBytesSpilled": [
              3.14,
            ],
            "outputBytes": [
              3.14,
            ],
            "outputRecords": [
              3.14,
            ],
            "peakMemoryMetrics": {
              "executorMetrics": [
                {
                  "metrics": {
                    "a_key": "A String",
                  },
                },
              ],
              "quantiles": [
                3.14,
              ],
            },
            "quantiles": [
              3.14,
            ],
            "shuffleRead": [
              3.14,
            ],
            "shuffleReadRecords": [
              3.14,
            ],
            "shuffleWrite": [
              3.14,
            ],
            "shuffleWriteRecords": [
              3.14,
            ],
            "succeededTasks": [
              3.14,
            ],
            "taskTimeMillis": [
              3.14,
            ],
          },
          "executorSummary": {
            "a_key": { # Executor resources consumed by a stage.
              "diskBytesSpilled": "A String",
              "executorId": "A String",
              "failedTasks": 42,
              "inputBytes": "A String",
              "inputRecords": "A String",
              "isExcludedForStage": True or False,
              "killedTasks": 42,
              "memoryBytesSpilled": "A String",
              "outputBytes": "A String",
              "outputRecords": "A String",
              "peakMemoryMetrics": {
                "metrics": {
                  "a_key": "A String",
                },
              },
              "shuffleRead": "A String",
              "shuffleReadRecords": "A String",
              "shuffleWrite": "A String",
              "shuffleWriteRecords": "A String",
              "stageAttemptId": 42,
              "stageId": "A String",
              "succeededTasks": 42,
              "taskTimeMillis": "A String",
            },
          },
          "failureReason": "A String",
          "firstTaskLaunchedTime": "A String",
          "isShufflePushEnabled": True or False,
          "jobIds": [
            "A String",
          ],
          "killedTasksSummary": {
            "a_key": 42,
          },
          "locality": {
            "a_key": "A String",
          },
          "name": "A String",
          "numActiveTasks": 42,
          "numCompleteTasks": 42,
          "numCompletedIndices": 42,
          "numFailedTasks": 42,
          "numKilledTasks": 42,
          "numTasks": 42,
          "parentStageIds": [
            "A String",
          ],
          "peakExecutorMetrics": {
            "metrics": {
              "a_key": "A String",
            },
          },
          "rddIds": [
            "A String",
          ],
          "resourceProfileId": 42,
          "schedulingPool": "A String",
          "shuffleMergersCount": 42,
          "speculationSummary": { # Details of the speculation task when speculative execution is enabled.
            "numActiveTasks": 42,
            "numCompletedTasks": 42,
            "numFailedTasks": 42,
            "numKilledTasks": 42,
            "numTasks": 42,
            "stageAttemptId": 42,
            "stageId": "A String",
          },
          "stageAttemptId": 42,
          "stageId": "A String",
          "stageMetrics": { # Stage Level Aggregated Metrics
            "diskBytesSpilled": "A String",
            "executorCpuTimeNanos": "A String",
            "executorDeserializeCpuTimeNanos": "A String",
            "executorDeserializeTimeMillis": "A String",
            "executorRunTimeMillis": "A String",
            "jvmGcTimeMillis": "A String",
            "memoryBytesSpilled": "A String",
            "peakExecutionMemoryBytes": "A String",
            "resultSerializationTimeMillis": "A String",
            "resultSize": "A String",
            "stageInputMetrics": { # Metrics about the input read by the stage.
              "bytesRead": "A String",
              "recordsRead": "A String",
            },
            "stageOutputMetrics": { # Metrics about the output written by the stage.
              "bytesWritten": "A String",
              "recordsWritten": "A String",
            },
            "stageShuffleReadMetrics": { # Shuffle data read for the stage.
              "bytesRead": "A String",
              "fetchWaitTimeMillis": "A String",
              "localBlocksFetched": "A String",
              "localBytesRead": "A String",
              "recordsRead": "A String",
              "remoteBlocksFetched": "A String",
              "remoteBytesRead": "A String",
              "remoteBytesReadToDisk": "A String",
              "remoteReqsDuration": "A String",
              "stageShufflePushReadMetrics": {
                "corruptMergedBlockChunks": "A String",
                "localMergedBlocksFetched": "A String",
                "localMergedBytesRead": "A String",
                "localMergedChunksFetched": "A String",
                "mergedFetchFallbackCount": "A String",
                "remoteMergedBlocksFetched": "A String",
                "remoteMergedBytesRead": "A String",
                "remoteMergedChunksFetched": "A String",
                "remoteMergedReqsDuration": "A String",
              },
            },
            "stageShuffleWriteMetrics": { # Shuffle data written for the stage.
              "bytesWritten": "A String",
              "recordsWritten": "A String",
              "writeTimeNanos": "A String",
            },
          },
          "status": "A String",
          "submissionTime": "A String",
          "taskQuantileMetrics": { # Summary metrics fields. These are included in response only if present in summary_metrics_mask field in request
            "diskBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "durationMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorDeserializeCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorDeserializeTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorRunTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "gettingResultTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "inputMetrics": {
              "bytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "recordsRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "jvmGcTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "memoryBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "outputMetrics": {
              "bytesWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "recordsWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "peakExecutionMemoryBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "resultSerializationTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "resultSize": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "schedulerDelayMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "shuffleReadMetrics": {
              "fetchWaitTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "localBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "readBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "readRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteBytesReadToDisk": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "shufflePushReadMetrics": {
                "corruptMergedBlockChunks": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "mergedFetchFallbackCount": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
              },
              "totalBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "shuffleWriteMetrics": {
              "writeBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "writeRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "writeTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
          },
          "tasks": {
            "a_key": { # Data corresponding to tasks created by spark.
              "accumulatorUpdates": [
                {
                  "accumullableInfoId": "A String",
                  "name": "A String",
                  "update": "A String",
                  "value": "A String",
                },
              ],
              "attempt": 42,
              "durationMillis": "A String",
              "errorMessage": "A String",
              "executorId": "A String",
              "executorLogs": {
                "a_key": "A String",
              },
              "gettingResultTimeMillis": "A String",
              "hasMetrics": True or False,
              "host": "A String",
              "index": 42,
              "launchTime": "A String",
              "partitionId": 42,
              "resultFetchStart": "A String",
              "schedulerDelayMillis": "A String",
              "speculative": True or False,
              "stageAttemptId": 42,
              "stageId": "A String",
              "status": "A String",
              "taskId": "A String",
              "taskLocality": "A String",
              "taskMetrics": { # Executor Task Metrics
                "diskBytesSpilled": "A String",
                "executorCpuTimeNanos": "A String",
                "executorDeserializeCpuTimeNanos": "A String",
                "executorDeserializeTimeMillis": "A String",
                "executorRunTimeMillis": "A String",
                "inputMetrics": { # Metrics about the input data read by the task.
                  "bytesRead": "A String",
                  "recordsRead": "A String",
                },
                "jvmGcTimeMillis": "A String",
                "memoryBytesSpilled": "A String",
                "outputMetrics": { # Metrics about the data written by the task.
                  "bytesWritten": "A String",
                  "recordsWritten": "A String",
                },
                "peakExecutionMemoryBytes": "A String",
                "resultSerializationTimeMillis": "A String",
                "resultSize": "A String",
                "shuffleReadMetrics": { # Shuffle data read by the task.
                  "fetchWaitTimeMillis": "A String",
                  "localBlocksFetched": "A String",
                  "localBytesRead": "A String",
                  "recordsRead": "A String",
                  "remoteBlocksFetched": "A String",
                  "remoteBytesRead": "A String",
                  "remoteBytesReadToDisk": "A String",
                  "remoteReqsDuration": "A String",
                  "shufflePushReadMetrics": {
                    "corruptMergedBlockChunks": "A String",
                    "localMergedBlocksFetched": "A String",
                    "localMergedBytesRead": "A String",
                    "localMergedChunksFetched": "A String",
                    "mergedFetchFallbackCount": "A String",
                    "remoteMergedBlocksFetched": "A String",
                    "remoteMergedBytesRead": "A String",
                    "remoteMergedChunksFetched": "A String",
                    "remoteMergedReqsDuration": "A String",
                  },
                },
                "shuffleWriteMetrics": { # Shuffle data written by task.
                  "bytesWritten": "A String",
                  "recordsWritten": "A String",
                  "writeTimeNanos": "A String",
                },
              },
            },
          },
        },
      ],
    }

`searchStageAttempts_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`searchStages(name, pageSize=None, pageToken=None, parent=None, stageStatus=None, summaryMetricsMask=None, x__xgafv=None)`

    Obtain data corresponding to stages for a Spark Application.

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      pageSize: integer, Optional. Maximum number of stages (paging based on stage_id) to return in each response. The service may return fewer than this. The default page size is 10; the maximum page size is 100.
      pageToken: string, Optional. A page token received from a previous FetchSparkApplicationStagesList call. Provide this token to retrieve the subsequent page.
      parent: string, Required. Parent (Batch) resource reference.
      stageStatus: string, Optional. List only stages in the given state.
        Allowed values
          STAGE_STATUS_UNSPECIFIED -
          STAGE_STATUS_ACTIVE -
          STAGE_STATUS_COMPLETE -
          STAGE_STATUS_FAILED -
          STAGE_STATUS_PENDING -
          STAGE_STATUS_SKIPPED -
      summaryMetricsMask: string, Optional. The list of summary metrics fields to include. Empty list will default to skip all summary metrics fields. Example, if the response should include TaskQuantileMetrics, the request should have task_quantile_metrics in summary_metrics_mask field
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # A list of stages associated with a Spark Application.
      "nextPageToken": "A String", # This token is included in the response if there are more results to fetch. To fetch additional results, provide this value as the page_token in a subsequent SearchSparkApplicationStages.
      "sparkApplicationStages": [ # Output only. Data corresponding to a stage.
        { # Data corresponding to a stage.
          "accumulatorUpdates": [
            {
              "accumullableInfoId": "A String",
              "name": "A String",
              "update": "A String",
              "value": "A String",
            },
          ],
          "completionTime": "A String",
          "description": "A String",
          "details": "A String",
          "executorMetricsDistributions": {
            "diskBytesSpilled": [
              3.14,
            ],
            "failedTasks": [
              3.14,
            ],
            "inputBytes": [
              3.14,
            ],
            "inputRecords": [
              3.14,
            ],
            "killedTasks": [
              3.14,
            ],
            "memoryBytesSpilled": [
              3.14,
            ],
            "outputBytes": [
              3.14,
            ],
            "outputRecords": [
              3.14,
            ],
            "peakMemoryMetrics": {
              "executorMetrics": [
                {
                  "metrics": {
                    "a_key": "A String",
                  },
                },
              ],
              "quantiles": [
                3.14,
              ],
            },
            "quantiles": [
              3.14,
            ],
            "shuffleRead": [
              3.14,
            ],
            "shuffleReadRecords": [
              3.14,
            ],
            "shuffleWrite": [
              3.14,
            ],
            "shuffleWriteRecords": [
              3.14,
            ],
            "succeededTasks": [
              3.14,
            ],
            "taskTimeMillis": [
              3.14,
            ],
          },
          "executorSummary": {
            "a_key": { # Executor resources consumed by a stage.
              "diskBytesSpilled": "A String",
              "executorId": "A String",
              "failedTasks": 42,
              "inputBytes": "A String",
              "inputRecords": "A String",
              "isExcludedForStage": True or False,
              "killedTasks": 42,
              "memoryBytesSpilled": "A String",
              "outputBytes": "A String",
              "outputRecords": "A String",
              "peakMemoryMetrics": {
                "metrics": {
                  "a_key": "A String",
                },
              },
              "shuffleRead": "A String",
              "shuffleReadRecords": "A String",
              "shuffleWrite": "A String",
              "shuffleWriteRecords": "A String",
              "stageAttemptId": 42,
              "stageId": "A String",
              "succeededTasks": 42,
              "taskTimeMillis": "A String",
            },
          },
          "failureReason": "A String",
          "firstTaskLaunchedTime": "A String",
          "isShufflePushEnabled": True or False,
          "jobIds": [
            "A String",
          ],
          "killedTasksSummary": {
            "a_key": 42,
          },
          "locality": {
            "a_key": "A String",
          },
          "name": "A String",
          "numActiveTasks": 42,
          "numCompleteTasks": 42,
          "numCompletedIndices": 42,
          "numFailedTasks": 42,
          "numKilledTasks": 42,
          "numTasks": 42,
          "parentStageIds": [
            "A String",
          ],
          "peakExecutorMetrics": {
            "metrics": {
              "a_key": "A String",
            },
          },
          "rddIds": [
            "A String",
          ],
          "resourceProfileId": 42,
          "schedulingPool": "A String",
          "shuffleMergersCount": 42,
          "speculationSummary": { # Details of the speculation task when speculative execution is enabled.
            "numActiveTasks": 42,
            "numCompletedTasks": 42,
            "numFailedTasks": 42,
            "numKilledTasks": 42,
            "numTasks": 42,
            "stageAttemptId": 42,
            "stageId": "A String",
          },
          "stageAttemptId": 42,
          "stageId": "A String",
          "stageMetrics": { # Stage Level Aggregated Metrics
            "diskBytesSpilled": "A String",
            "executorCpuTimeNanos": "A String",
            "executorDeserializeCpuTimeNanos": "A String",
            "executorDeserializeTimeMillis": "A String",
            "executorRunTimeMillis": "A String",
            "jvmGcTimeMillis": "A String",
            "memoryBytesSpilled": "A String",
            "peakExecutionMemoryBytes": "A String",
            "resultSerializationTimeMillis": "A String",
            "resultSize": "A String",
            "stageInputMetrics": { # Metrics about the input read by the stage.
              "bytesRead": "A String",
              "recordsRead": "A String",
            },
            "stageOutputMetrics": { # Metrics about the output written by the stage.
              "bytesWritten": "A String",
              "recordsWritten": "A String",
            },
            "stageShuffleReadMetrics": { # Shuffle data read for the stage.
              "bytesRead": "A String",
              "fetchWaitTimeMillis": "A String",
              "localBlocksFetched": "A String",
              "localBytesRead": "A String",
              "recordsRead": "A String",
              "remoteBlocksFetched": "A String",
              "remoteBytesRead": "A String",
              "remoteBytesReadToDisk": "A String",
              "remoteReqsDuration": "A String",
              "stageShufflePushReadMetrics": {
                "corruptMergedBlockChunks": "A String",
                "localMergedBlocksFetched": "A String",
                "localMergedBytesRead": "A String",
                "localMergedChunksFetched": "A String",
                "mergedFetchFallbackCount": "A String",
                "remoteMergedBlocksFetched": "A String",
                "remoteMergedBytesRead": "A String",
                "remoteMergedChunksFetched": "A String",
                "remoteMergedReqsDuration": "A String",
              },
            },
            "stageShuffleWriteMetrics": { # Shuffle data written for the stage.
              "bytesWritten": "A String",
              "recordsWritten": "A String",
              "writeTimeNanos": "A String",
            },
          },
          "status": "A String",
          "submissionTime": "A String",
          "taskQuantileMetrics": { # Summary metrics fields. These are included in response only if present in summary_metrics_mask field in request
            "diskBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "durationMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorDeserializeCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorDeserializeTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "executorRunTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "gettingResultTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "inputMetrics": {
              "bytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "recordsRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "jvmGcTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "memoryBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "outputMetrics": {
              "bytesWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "recordsWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "peakExecutionMemoryBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "resultSerializationTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "resultSize": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "schedulerDelayMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
              "count": "A String",
              "maximum": "A String",
              "minimum": "A String",
              "percentile25": "A String",
              "percentile50": "A String",
              "percentile75": "A String",
              "sum": "A String",
            },
            "shuffleReadMetrics": {
              "fetchWaitTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "localBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "readBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "readRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteBytesReadToDisk": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "remoteReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "shufflePushReadMetrics": {
                "corruptMergedBlockChunks": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "mergedFetchFallbackCount": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteMergedReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
              },
              "totalBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
            "shuffleWriteMetrics": {
              "writeBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "writeRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "writeTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
            },
          },
          "tasks": {
            "a_key": { # Data corresponding to tasks created by spark.
              "accumulatorUpdates": [
                {
                  "accumullableInfoId": "A String",
                  "name": "A String",
                  "update": "A String",
                  "value": "A String",
                },
              ],
              "attempt": 42,
              "durationMillis": "A String",
              "errorMessage": "A String",
              "executorId": "A String",
              "executorLogs": {
                "a_key": "A String",
              },
              "gettingResultTimeMillis": "A String",
              "hasMetrics": True or False,
              "host": "A String",
              "index": 42,
              "launchTime": "A String",
              "partitionId": 42,
              "resultFetchStart": "A String",
              "schedulerDelayMillis": "A String",
              "speculative": True or False,
              "stageAttemptId": 42,
              "stageId": "A String",
              "status": "A String",
              "taskId": "A String",
              "taskLocality": "A String",
              "taskMetrics": { # Executor Task Metrics
                "diskBytesSpilled": "A String",
                "executorCpuTimeNanos": "A String",
                "executorDeserializeCpuTimeNanos": "A String",
                "executorDeserializeTimeMillis": "A String",
                "executorRunTimeMillis": "A String",
                "inputMetrics": { # Metrics about the input data read by the task.
                  "bytesRead": "A String",
                  "recordsRead": "A String",
                },
                "jvmGcTimeMillis": "A String",
                "memoryBytesSpilled": "A String",
                "outputMetrics": { # Metrics about the data written by the task.
                  "bytesWritten": "A String",
                  "recordsWritten": "A String",
                },
                "peakExecutionMemoryBytes": "A String",
                "resultSerializationTimeMillis": "A String",
                "resultSize": "A String",
                "shuffleReadMetrics": { # Shuffle data read by the task.
                  "fetchWaitTimeMillis": "A String",
                  "localBlocksFetched": "A String",
                  "localBytesRead": "A String",
                  "recordsRead": "A String",
                  "remoteBlocksFetched": "A String",
                  "remoteBytesRead": "A String",
                  "remoteBytesReadToDisk": "A String",
                  "remoteReqsDuration": "A String",
                  "shufflePushReadMetrics": {
                    "corruptMergedBlockChunks": "A String",
                    "localMergedBlocksFetched": "A String",
                    "localMergedBytesRead": "A String",
                    "localMergedChunksFetched": "A String",
                    "mergedFetchFallbackCount": "A String",
                    "remoteMergedBlocksFetched": "A String",
                    "remoteMergedBytesRead": "A String",
                    "remoteMergedChunksFetched": "A String",
                    "remoteMergedReqsDuration": "A String",
                  },
                },
                "shuffleWriteMetrics": { # Shuffle data written by task.
                  "bytesWritten": "A String",
                  "recordsWritten": "A String",
                  "writeTimeNanos": "A String",
                },
              },
            },
          },
        },
      ],
    }

`searchStages_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`search_next()`

    Retrieves the next page of results.

            Args:
              previous_request: The request for the previous page. (required)
              previous_response: The response from the request for the previous page. (required)

            Returns:
              A request object that you can call 'execute()' on to request the next
              page. Returns None if there are no more items in the collection.

`summarizeExecutors(name, parent=None, x__xgafv=None)`

    Obtain summary of Executor Summary for a Spark Application

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Consolidated summary of executors for a Spark Application.
      "activeExecutorSummary": { # Consolidated summary about executors used by the application. # Consolidated summary for active executors.
        "activeTasks": 42,
        "completedTasks": 42,
        "count": 42,
        "diskUsed": "A String",
        "failedTasks": 42,
        "isExcluded": 42,
        "maxMemory": "A String",
        "memoryMetrics": {
          "totalOffHeapStorageMemory": "A String",
          "totalOnHeapStorageMemory": "A String",
          "usedOffHeapStorageMemory": "A String",
          "usedOnHeapStorageMemory": "A String",
        },
        "memoryUsed": "A String",
        "rddBlocks": 42,
        "totalCores": 42,
        "totalDurationMillis": "A String",
        "totalGcTimeMillis": "A String",
        "totalInputBytes": "A String",
        "totalShuffleRead": "A String",
        "totalShuffleWrite": "A String",
        "totalTasks": 42,
      },
      "applicationId": "A String", # Spark Application Id
      "deadExecutorSummary": { # Consolidated summary about executors used by the application. # Consolidated summary for dead executors.
        "activeTasks": 42,
        "completedTasks": 42,
        "count": 42,
        "diskUsed": "A String",
        "failedTasks": 42,
        "isExcluded": 42,
        "maxMemory": "A String",
        "memoryMetrics": {
          "totalOffHeapStorageMemory": "A String",
          "totalOnHeapStorageMemory": "A String",
          "usedOffHeapStorageMemory": "A String",
          "usedOnHeapStorageMemory": "A String",
        },
        "memoryUsed": "A String",
        "rddBlocks": 42,
        "totalCores": 42,
        "totalDurationMillis": "A String",
        "totalGcTimeMillis": "A String",
        "totalInputBytes": "A String",
        "totalShuffleRead": "A String",
        "totalShuffleWrite": "A String",
        "totalTasks": 42,
      },
      "totalExecutorSummary": { # Consolidated summary about executors used by the application. # Overall consolidated summary for all executors.
        "activeTasks": 42,
        "completedTasks": 42,
        "count": 42,
        "diskUsed": "A String",
        "failedTasks": 42,
        "isExcluded": 42,
        "maxMemory": "A String",
        "memoryMetrics": {
          "totalOffHeapStorageMemory": "A String",
          "totalOnHeapStorageMemory": "A String",
          "usedOffHeapStorageMemory": "A String",
          "usedOnHeapStorageMemory": "A String",
        },
        "memoryUsed": "A String",
        "rddBlocks": 42,
        "totalCores": 42,
        "totalDurationMillis": "A String",
        "totalGcTimeMillis": "A String",
        "totalInputBytes": "A String",
        "totalShuffleRead": "A String",
        "totalShuffleWrite": "A String",
        "totalTasks": 42,
      },
    }

`summarizeJobs(name, parent=None, x__xgafv=None)`

    Obtain summary of Jobs for a Spark Application

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Summary of a Spark Application jobs.
      "jobsSummary": { # Data related to Jobs page summary # Summary of a Spark Application Jobs
        "activeJobs": 42, # Number of active jobs
        "applicationId": "A String", # Spark Application Id
        "attempts": [ # Attempts info
          { # Specific attempt of an application.
            "appSparkVersion": "A String",
            "attemptId": "A String",
            "completed": True or False,
            "durationMillis": "A String",
            "endTime": "A String",
            "lastUpdated": "A String",
            "sparkUser": "A String",
            "startTime": "A String",
          },
        ],
        "completedJobs": 42, # Number of completed jobs
        "failedJobs": 42, # Number of failed jobs
        "schedulingMode": "A String", # Spark Scheduling mode
      },
    }

`summarizeStageAttemptTasks(name, parent=None, stageAttemptId=None, stageId=None, x__xgafv=None)`

    Obtain summary of Tasks for a Spark Application Stage Attempt

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      stageAttemptId: integer, Required. Stage Attempt ID
      stageId: string, Required. Stage ID
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Summary of tasks for a Spark Application stage attempt.
      "stageAttemptTasksSummary": { # Data related to tasks summary for a Spark Stage Attempt # Summary of tasks for a Spark Application Stage Attempt
        "applicationId": "A String",
        "numFailedTasks": 42,
        "numKilledTasks": 42,
        "numPendingTasks": 42,
        "numRunningTasks": 42,
        "numSuccessTasks": 42,
        "numTasks": 42,
        "stageAttemptId": 42,
        "stageId": "A String",
      },
    }

`summarizeStages(name, parent=None, x__xgafv=None)`

    Obtain summary of Stages for a Spark Application

    Args:
      name: string, Required. The fully qualified name of the batch to retrieve in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      parent: string, Required. Parent (Batch) resource reference.
      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Summary of a Spark Application stages.
      "stagesSummary": { # Data related to Stages page summary # Summary of a Spark Application Stages
        "applicationId": "A String",
        "numActiveStages": 42,
        "numCompletedStages": 42,
        "numFailedStages": 42,
        "numPendingStages": 42,
        "numSkippedStages": 42,
      },
    }

`write(name, body=None, x__xgafv=None)`

    Write wrapper objects from dataplane to spanner

    Args:
      name: string, Required. The fully qualified name of the spark application to write data about in the format "projects/PROJECT_ID/locations/DATAPROC_REGION/batches/BATCH_ID/sparkApplications/APPLICATION_ID" (required)
      body: object, The request body.
        The object takes the form of:

    { # Write Spark Application data to internal storage systems
      "parent": "A String", # Required. Parent (Batch) resource reference.
      "sparkWrapperObjects": [
        { # Outer message that contains the data obtained from spark listener, packaged with information that is required to process it.
          "appSummary": {
            "numCompletedJobs": 42,
            "numCompletedStages": 42,
          },
          "applicationEnvironmentInfo": { # Details about the Environment that the application is running in.
            "classpathEntries": {
              "a_key": "A String",
            },
            "hadoopProperties": {
              "a_key": "A String",
            },
            "metricsProperties": {
              "a_key": "A String",
            },
            "resourceProfiles": [
              { # Resource profile that contains information about all the resources required by executors and tasks.
                "executorResources": {
                  "a_key": { # Resources used per executor used by the application.
                    "amount": "A String",
                    "discoveryScript": "A String",
                    "resourceName": "A String",
                    "vendor": "A String",
                  },
                },
                "resourceProfileId": 42,
                "taskResources": {
                  "a_key": { # Resources used per task created by the application.
                    "amount": 3.14,
                    "resourceName": "A String",
                  },
                },
              },
            ],
            "runtime": {
              "javaHome": "A String",
              "javaVersion": "A String",
              "scalaVersion": "A String",
            },
            "sparkProperties": {
              "a_key": "A String",
            },
            "systemProperties": {
              "a_key": "A String",
            },
          },
          "applicationId": "A String", # Application Id created by Spark.
          "applicationInfo": { # High level information corresponding to an application.
            "applicationContextIngestionStatus": "A String",
            "applicationId": "A String",
            "attempts": [
              { # Specific attempt of an application.
                "appSparkVersion": "A String",
                "attemptId": "A String",
                "completed": True or False,
                "durationMillis": "A String",
                "endTime": "A String",
                "lastUpdated": "A String",
                "sparkUser": "A String",
                "startTime": "A String",
              },
            ],
            "coresGranted": 42,
            "coresPerExecutor": 42,
            "maxCores": 42,
            "memoryPerExecutorMb": 42,
            "name": "A String",
            "quantileDataStatus": "A String",
          },
          "eventTimestamp": "A String", # VM Timestamp associated with the data object.
          "executorStageSummary": { # Executor resources consumed by a stage.
            "diskBytesSpilled": "A String",
            "executorId": "A String",
            "failedTasks": 42,
            "inputBytes": "A String",
            "inputRecords": "A String",
            "isExcludedForStage": True or False,
            "killedTasks": 42,
            "memoryBytesSpilled": "A String",
            "outputBytes": "A String",
            "outputRecords": "A String",
            "peakMemoryMetrics": {
              "metrics": {
                "a_key": "A String",
              },
            },
            "shuffleRead": "A String",
            "shuffleReadRecords": "A String",
            "shuffleWrite": "A String",
            "shuffleWriteRecords": "A String",
            "stageAttemptId": 42,
            "stageId": "A String",
            "succeededTasks": 42,
            "taskTimeMillis": "A String",
          },
          "executorSummary": { # Details about executors used by the application.
            "activeTasks": 42,
            "addTime": "A String",
            "attributes": {
              "a_key": "A String",
            },
            "completedTasks": 42,
            "diskUsed": "A String",
            "excludedInStages": [
              "A String",
            ],
            "executorId": "A String",
            "executorLogs": {
              "a_key": "A String",
            },
            "failedTasks": 42,
            "hostPort": "A String",
            "isActive": True or False,
            "isExcluded": True or False,
            "maxMemory": "A String",
            "maxTasks": 42,
            "memoryMetrics": {
              "totalOffHeapStorageMemory": "A String",
              "totalOnHeapStorageMemory": "A String",
              "usedOffHeapStorageMemory": "A String",
              "usedOnHeapStorageMemory": "A String",
            },
            "memoryUsed": "A String",
            "peakMemoryMetrics": {
              "metrics": {
                "a_key": "A String",
              },
            },
            "rddBlocks": 42,
            "removeReason": "A String",
            "removeTime": "A String",
            "resourceProfileId": 42,
            "resources": {
              "a_key": {
                "addresses": [
                  "A String",
                ],
                "name": "A String",
              },
            },
            "totalCores": 42,
            "totalDurationMillis": "A String",
            "totalGcTimeMillis": "A String",
            "totalInputBytes": "A String",
            "totalShuffleRead": "A String",
            "totalShuffleWrite": "A String",
            "totalTasks": 42,
          },
          "jobData": { # Data corresponding to a spark job.
            "completionTime": "A String",
            "description": "A String",
            "jobGroup": "A String",
            "jobId": "A String",
            "killTasksSummary": {
              "a_key": 42,
            },
            "name": "A String",
            "numActiveStages": 42,
            "numActiveTasks": 42,
            "numCompletedIndices": 42,
            "numCompletedStages": 42,
            "numCompletedTasks": 42,
            "numFailedStages": 42,
            "numFailedTasks": 42,
            "numKilledTasks": 42,
            "numSkippedStages": 42,
            "numSkippedTasks": 42,
            "numTasks": 42,
            "skippedStages": [
              42,
            ],
            "sqlExecutionId": "A String",
            "stageIds": [
              "A String",
            ],
            "status": "A String",
            "submissionTime": "A String",
          },
          "nativeBuildInfoUiData": { # Native Build Info
            "buildClass": "A String", # Optional. Build class of Native.
            "buildInfo": [ # Optional. Build related details.
              { # Native Build Info
                "buildKey": "A String", # Optional. Build key.
                "buildValue": "A String", # Optional. Build value.
              },
            ],
          },
          "nativeSqlExecutionUiData": { # Native SQL Execution Data # Native SQL Execution Info
            "description": "A String", # Optional. Description of the execution.
            "executionId": "A String", # Required. Execution ID of the Native SQL Execution.
            "fallbackDescription": "A String", # Optional. Description of the fallback.
            "fallbackNodeToReason": [ # Optional. Fallback node to reason.
              { # Native SQL Execution Data
                "fallbackNode": "A String", # Optional. Fallback node information.
                "fallbackReason": "A String", # Optional. Fallback to Spark reason.
              },
            ],
            "numFallbackNodes": 42, # Optional. Number of nodes fallen back to Spark.
            "numNativeNodes": 42, # Optional. Number of nodes in Native.
          },
          "poolData": { # Pool Data
            "name": "A String",
            "stageIds": [
              "A String",
            ],
          },
          "processSummary": { # Process Summary
            "addTime": "A String",
            "hostPort": "A String",
            "isActive": True or False,
            "processId": "A String",
            "processLogs": {
              "a_key": "A String",
            },
            "removeTime": "A String",
            "totalCores": 42,
          },
          "rddOperationGraph": { # Graph representing RDD dependencies. Consists of edges and a root cluster.
            "edges": [
              { # A directed edge representing dependency between two RDDs.
                "fromId": 42,
                "toId": 42,
              },
            ],
            "incomingEdges": [
              { # A directed edge representing dependency between two RDDs.
                "fromId": 42,
                "toId": 42,
              },
            ],
            "outgoingEdges": [
              { # A directed edge representing dependency between two RDDs.
                "fromId": 42,
                "toId": 42,
              },
            ],
            "rootCluster": { # A grouping of nodes representing higher level constructs (stage, job etc.).
              "childClusters": [
                # Object with schema name: RddOperationCluster
              ],
              "childNodes": [
                { # A node in the RDD operation graph. Corresponds to a single RDD.
                  "barrier": True or False,
                  "cached": True or False,
                  "callsite": "A String",
                  "name": "A String",
                  "nodeId": 42,
                  "outputDeterministicLevel": "A String",
                },
              ],
              "name": "A String",
              "rddClusterId": "A String",
            },
            "stageId": "A String",
          },
          "rddStorageInfo": { # Overall data about RDD storage.
            "dataDistribution": [
              { # Details about RDD usage.
                "address": "A String",
                "diskUsed": "A String",
                "memoryRemaining": "A String",
                "memoryUsed": "A String",
                "offHeapMemoryRemaining": "A String",
                "offHeapMemoryUsed": "A String",
                "onHeapMemoryRemaining": "A String",
                "onHeapMemoryUsed": "A String",
              },
            ],
            "diskUsed": "A String",
            "memoryUsed": "A String",
            "name": "A String",
            "numCachedPartitions": 42,
            "numPartitions": 42,
            "partitions": [
              { # Information about RDD partitions.
                "blockName": "A String",
                "diskUsed": "A String",
                "executors": [
                  "A String",
                ],
                "memoryUsed": "A String",
                "storageLevel": "A String",
              },
            ],
            "rddStorageId": 42,
            "storageLevel": "A String",
          },
          "resourceProfileInfo": { # Resource profile that contains information about all the resources required by executors and tasks.
            "executorResources": {
              "a_key": { # Resources used per executor used by the application.
                "amount": "A String",
                "discoveryScript": "A String",
                "resourceName": "A String",
                "vendor": "A String",
              },
            },
            "resourceProfileId": 42,
            "taskResources": {
              "a_key": { # Resources used per task created by the application.
                "amount": 3.14,
                "resourceName": "A String",
              },
            },
          },
          "sparkPlanGraph": { # A graph used for storing information of an executionPlan of DataFrame.
            "edges": [
              { # Represents a directed edge in the spark plan tree from child to parent.
                "fromId": "A String",
                "toId": "A String",
              },
            ],
            "executionId": "A String",
            "nodes": [
              { # Wrapper user to represent either a node or a cluster.
                "cluster": { # Represents a tree of spark plan.
                  "desc": "A String",
                  "metrics": [
                    { # Metrics related to SQL execution.
                      "accumulatorId": "A String",
                      "metricType": "A String",
                      "name": "A String",
                    },
                  ],
                  "name": "A String",
                  "nodes": [
                    # Object with schema name: SparkPlanGraphNodeWrapper
                  ],
                  "sparkPlanGraphClusterId": "A String",
                },
                "node": { # Represents a node in the spark plan tree.
                  "desc": "A String",
                  "metrics": [
                    { # Metrics related to SQL execution.
                      "accumulatorId": "A String",
                      "metricType": "A String",
                      "name": "A String",
                    },
                  ],
                  "name": "A String",
                  "sparkPlanGraphNodeId": "A String",
                },
              },
            ],
          },
          "speculationStageSummary": { # Details of the speculation task when speculative execution is enabled.
            "numActiveTasks": 42,
            "numCompletedTasks": 42,
            "numFailedTasks": 42,
            "numKilledTasks": 42,
            "numTasks": 42,
            "stageAttemptId": 42,
            "stageId": "A String",
          },
          "sqlExecutionUiData": { # SQL Execution Data
            "completionTime": "A String",
            "description": "A String",
            "details": "A String",
            "errorMessage": "A String",
            "executionId": "A String",
            "jobs": {
              "a_key": "A String",
            },
            "metricValues": {
              "a_key": "A String",
            },
            "metricValuesIsNull": True or False,
            "metrics": [
              { # Metrics related to SQL execution.
                "accumulatorId": "A String",
                "metricType": "A String",
                "name": "A String",
              },
            ],
            "modifiedConfigs": {
              "a_key": "A String",
            },
            "physicalPlanDescription": "A String",
            "rootExecutionId": "A String",
            "stages": [
              "A String",
            ],
            "submissionTime": "A String",
          },
          "stageData": { # Data corresponding to a stage.
            "accumulatorUpdates": [
              {
                "accumullableInfoId": "A String",
                "name": "A String",
                "update": "A String",
                "value": "A String",
              },
            ],
            "completionTime": "A String",
            "description": "A String",
            "details": "A String",
            "executorMetricsDistributions": {
              "diskBytesSpilled": [
                3.14,
              ],
              "failedTasks": [
                3.14,
              ],
              "inputBytes": [
                3.14,
              ],
              "inputRecords": [
                3.14,
              ],
              "killedTasks": [
                3.14,
              ],
              "memoryBytesSpilled": [
                3.14,
              ],
              "outputBytes": [
                3.14,
              ],
              "outputRecords": [
                3.14,
              ],
              "peakMemoryMetrics": {
                "executorMetrics": [
                  {
                    "metrics": {
                      "a_key": "A String",
                    },
                  },
                ],
                "quantiles": [
                  3.14,
                ],
              },
              "quantiles": [
                3.14,
              ],
              "shuffleRead": [
                3.14,
              ],
              "shuffleReadRecords": [
                3.14,
              ],
              "shuffleWrite": [
                3.14,
              ],
              "shuffleWriteRecords": [
                3.14,
              ],
              "succeededTasks": [
                3.14,
              ],
              "taskTimeMillis": [
                3.14,
              ],
            },
            "executorSummary": {
              "a_key": { # Executor resources consumed by a stage.
                "diskBytesSpilled": "A String",
                "executorId": "A String",
                "failedTasks": 42,
                "inputBytes": "A String",
                "inputRecords": "A String",
                "isExcludedForStage": True or False,
                "killedTasks": 42,
                "memoryBytesSpilled": "A String",
                "outputBytes": "A String",
                "outputRecords": "A String",
                "peakMemoryMetrics": {
                  "metrics": {
                    "a_key": "A String",
                  },
                },
                "shuffleRead": "A String",
                "shuffleReadRecords": "A String",
                "shuffleWrite": "A String",
                "shuffleWriteRecords": "A String",
                "stageAttemptId": 42,
                "stageId": "A String",
                "succeededTasks": 42,
                "taskTimeMillis": "A String",
              },
            },
            "failureReason": "A String",
            "firstTaskLaunchedTime": "A String",
            "isShufflePushEnabled": True or False,
            "jobIds": [
              "A String",
            ],
            "killedTasksSummary": {
              "a_key": 42,
            },
            "locality": {
              "a_key": "A String",
            },
            "name": "A String",
            "numActiveTasks": 42,
            "numCompleteTasks": 42,
            "numCompletedIndices": 42,
            "numFailedTasks": 42,
            "numKilledTasks": 42,
            "numTasks": 42,
            "parentStageIds": [
              "A String",
            ],
            "peakExecutorMetrics": {
              "metrics": {
                "a_key": "A String",
              },
            },
            "rddIds": [
              "A String",
            ],
            "resourceProfileId": 42,
            "schedulingPool": "A String",
            "shuffleMergersCount": 42,
            "speculationSummary": { # Details of the speculation task when speculative execution is enabled.
              "numActiveTasks": 42,
              "numCompletedTasks": 42,
              "numFailedTasks": 42,
              "numKilledTasks": 42,
              "numTasks": 42,
              "stageAttemptId": 42,
              "stageId": "A String",
            },
            "stageAttemptId": 42,
            "stageId": "A String",
            "stageMetrics": { # Stage Level Aggregated Metrics
              "diskBytesSpilled": "A String",
              "executorCpuTimeNanos": "A String",
              "executorDeserializeCpuTimeNanos": "A String",
              "executorDeserializeTimeMillis": "A String",
              "executorRunTimeMillis": "A String",
              "jvmGcTimeMillis": "A String",
              "memoryBytesSpilled": "A String",
              "peakExecutionMemoryBytes": "A String",
              "resultSerializationTimeMillis": "A String",
              "resultSize": "A String",
              "stageInputMetrics": { # Metrics about the input read by the stage.
                "bytesRead": "A String",
                "recordsRead": "A String",
              },
              "stageOutputMetrics": { # Metrics about the output written by the stage.
                "bytesWritten": "A String",
                "recordsWritten": "A String",
              },
              "stageShuffleReadMetrics": { # Shuffle data read for the stage.
                "bytesRead": "A String",
                "fetchWaitTimeMillis": "A String",
                "localBlocksFetched": "A String",
                "localBytesRead": "A String",
                "recordsRead": "A String",
                "remoteBlocksFetched": "A String",
                "remoteBytesRead": "A String",
                "remoteBytesReadToDisk": "A String",
                "remoteReqsDuration": "A String",
                "stageShufflePushReadMetrics": {
                  "corruptMergedBlockChunks": "A String",
                  "localMergedBlocksFetched": "A String",
                  "localMergedBytesRead": "A String",
                  "localMergedChunksFetched": "A String",
                  "mergedFetchFallbackCount": "A String",
                  "remoteMergedBlocksFetched": "A String",
                  "remoteMergedBytesRead": "A String",
                  "remoteMergedChunksFetched": "A String",
                  "remoteMergedReqsDuration": "A String",
                },
              },
              "stageShuffleWriteMetrics": { # Shuffle data written for the stage.
                "bytesWritten": "A String",
                "recordsWritten": "A String",
                "writeTimeNanos": "A String",
              },
            },
            "status": "A String",
            "submissionTime": "A String",
            "taskQuantileMetrics": { # Summary metrics fields. These are included in response only if present in summary_metrics_mask field in request
              "diskBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "durationMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "executorCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "executorDeserializeCpuTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "executorDeserializeTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "executorRunTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "gettingResultTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "inputMetrics": {
                "bytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "recordsRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
              },
              "jvmGcTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "memoryBytesSpilled": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "outputMetrics": {
                "bytesWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "recordsWritten": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
              },
              "peakExecutionMemoryBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "resultSerializationTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "resultSize": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "schedulerDelayMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                "count": "A String",
                "maximum": "A String",
                "minimum": "A String",
                "percentile25": "A String",
                "percentile50": "A String",
                "percentile75": "A String",
                "sum": "A String",
              },
              "shuffleReadMetrics": {
                "fetchWaitTimeMillis": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "localBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "readBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "readRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteBytesReadToDisk": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "remoteReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "shufflePushReadMetrics": {
                  "corruptMergedBlockChunks": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "localMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "localMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "localMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "mergedFetchFallbackCount": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "remoteMergedBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "remoteMergedBytesRead": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "remoteMergedChunksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                  "remoteMergedReqsDuration": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                    "count": "A String",
                    "maximum": "A String",
                    "minimum": "A String",
                    "percentile25": "A String",
                    "percentile50": "A String",
                    "percentile75": "A String",
                    "sum": "A String",
                  },
                },
                "totalBlocksFetched": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
              },
              "shuffleWriteMetrics": {
                "writeBytes": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "writeRecords": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
                "writeTimeNanos": { # Quantile metrics data related to Tasks. Units can be seconds, bytes, milliseconds, etc depending on the message type.
                  "count": "A String",
                  "maximum": "A String",
                  "minimum": "A String",
                  "percentile25": "A String",
                  "percentile50": "A String",
                  "percentile75": "A String",
                  "sum": "A String",
                },
              },
            },
            "tasks": {
              "a_key": { # Data corresponding to tasks created by spark.
                "accumulatorUpdates": [
                  {
                    "accumullableInfoId": "A String",
                    "name": "A String",
                    "update": "A String",
                    "value": "A String",
                  },
                ],
                "attempt": 42,
                "durationMillis": "A String",
                "errorMessage": "A String",
                "executorId": "A String",
                "executorLogs": {
                  "a_key": "A String",
                },
                "gettingResultTimeMillis": "A String",
                "hasMetrics": True or False,
                "host": "A String",
                "index": 42,
                "launchTime": "A String",
                "partitionId": 42,
                "resultFetchStart": "A String",
                "schedulerDelayMillis": "A String",
                "speculative": True or False,
                "stageAttemptId": 42,
                "stageId": "A String",
                "status": "A String",
                "taskId": "A String",
                "taskLocality": "A String",
                "taskMetrics": { # Executor Task Metrics
                  "diskBytesSpilled": "A String",
                  "executorCpuTimeNanos": "A String",
                  "executorDeserializeCpuTimeNanos": "A String",
                  "executorDeserializeTimeMillis": "A String",
                  "executorRunTimeMillis": "A String",
                  "inputMetrics": { # Metrics about the input data read by the task.
                    "bytesRead": "A String",
                    "recordsRead": "A String",
                  },
                  "jvmGcTimeMillis": "A String",
                  "memoryBytesSpilled": "A String",
                  "outputMetrics": { # Metrics about the data written by the task.
                    "bytesWritten": "A String",
                    "recordsWritten": "A String",
                  },
                  "peakExecutionMemoryBytes": "A String",
                  "resultSerializationTimeMillis": "A String",
                  "resultSize": "A String",
                  "shuffleReadMetrics": { # Shuffle data read by the task.
                    "fetchWaitTimeMillis": "A String",
                    "localBlocksFetched": "A String",
                    "localBytesRead": "A String",
                    "recordsRead": "A String",
                    "remoteBlocksFetched": "A String",
                    "remoteBytesRead": "A String",
                    "remoteBytesReadToDisk": "A String",
                    "remoteReqsDuration": "A String",
                    "shufflePushReadMetrics": {
                      "corruptMergedBlockChunks": "A String",
                      "localMergedBlocksFetched": "A String",
                      "localMergedBytesRead": "A String",
                      "localMergedChunksFetched": "A String",
                      "mergedFetchFallbackCount": "A String",
                      "remoteMergedBlocksFetched": "A String",
                      "remoteMergedBytesRead": "A String",
                      "remoteMergedChunksFetched": "A String",
                      "remoteMergedReqsDuration": "A String",
                    },
                  },
                  "shuffleWriteMetrics": { # Shuffle data written by task.
                    "bytesWritten": "A String",
                    "recordsWritten": "A String",
                    "writeTimeNanos": "A String",
                  },
                },
              },
            },
          },
          "streamBlockData": { # Stream Block Data.
            "deserialized": True or False,
            "diskSize": "A String",
            "executorId": "A String",
            "hostPort": "A String",
            "memSize": "A String",
            "name": "A String",
            "storageLevel": "A String",
            "useDisk": True or False,
            "useMemory": True or False,
          },
          "streamingQueryData": { # Streaming
            "endTimestamp": "A String",
            "exception": "A String",
            "isActive": True or False,
            "name": "A String",
            "runId": "A String",
            "startTimestamp": "A String",
            "streamingQueryId": "A String",
          },
          "streamingQueryProgress": {
            "batchDuration": "A String",
            "batchId": "A String",
            "durationMillis": {
              "a_key": "A String",
            },
            "eventTime": {
              "a_key": "A String",
            },
            "name": "A String",
            "observedMetrics": {
              "a_key": "A String",
            },
            "runId": "A String",
            "sink": {
              "description": "A String",
              "metrics": {
                "a_key": "A String",
              },
              "numOutputRows": "A String",
            },
            "sources": [
              {
                "description": "A String",
                "endOffset": "A String",
                "inputRowsPerSecond": 3.14,
                "latestOffset": "A String",
                "metrics": {
                  "a_key": "A String",
                },
                "numInputRows": "A String",
                "processedRowsPerSecond": 3.14,
                "startOffset": "A String",
              },
            ],
            "stateOperators": [
              {
                "allRemovalsTimeMs": "A String",
                "allUpdatesTimeMs": "A String",
                "commitTimeMs": "A String",
                "customMetrics": {
                  "a_key": "A String",
                },
                "memoryUsedBytes": "A String",
                "numRowsDroppedByWatermark": "A String",
                "numRowsRemoved": "A String",
                "numRowsTotal": "A String",
                "numRowsUpdated": "A String",
                "numShufflePartitions": "A String",
                "numStateStoreInstances": "A String",
                "operatorName": "A String",
              },
            ],
            "streamingQueryProgressId": "A String",
            "timestamp": "A String",
          },
          "taskData": { # Data corresponding to tasks created by spark.
            "accumulatorUpdates": [
              {
                "accumullableInfoId": "A String",
                "name": "A String",
                "update": "A String",
                "value": "A String",
              },
            ],
            "attempt": 42,
            "durationMillis": "A String",
            "errorMessage": "A String",
            "executorId": "A String",
            "executorLogs": {
              "a_key": "A String",
            },
            "gettingResultTimeMillis": "A String",
            "hasMetrics": True or False,
            "host": "A String",
            "index": 42,
            "launchTime": "A String",
            "partitionId": 42,
            "resultFetchStart": "A String",
            "schedulerDelayMillis": "A String",
            "speculative": True or False,
            "stageAttemptId": 42,
            "stageId": "A String",
            "status": "A String",
            "taskId": "A String",
            "taskLocality": "A String",
            "taskMetrics": { # Executor Task Metrics
              "diskBytesSpilled": "A String",
              "executorCpuTimeNanos": "A String",
              "executorDeserializeCpuTimeNanos": "A String",
              "executorDeserializeTimeMillis": "A String",
              "executorRunTimeMillis": "A String",
              "inputMetrics": { # Metrics about the input data read by the task.
                "bytesRead": "A String",
                "recordsRead": "A String",
              },
              "jvmGcTimeMillis": "A String",
              "memoryBytesSpilled": "A String",
              "outputMetrics": { # Metrics about the data written by the task.
                "bytesWritten": "A String",
                "recordsWritten": "A String",
              },
              "peakExecutionMemoryBytes": "A String",
              "resultSerializationTimeMillis": "A String",
              "resultSize": "A String",
              "shuffleReadMetrics": { # Shuffle data read by the task.
                "fetchWaitTimeMillis": "A String",
                "localBlocksFetched": "A String",
                "localBytesRead": "A String",
                "recordsRead": "A String",
                "remoteBlocksFetched": "A String",
                "remoteBytesRead": "A String",
                "remoteBytesReadToDisk": "A String",
                "remoteReqsDuration": "A String",
                "shufflePushReadMetrics": {
                  "corruptMergedBlockChunks": "A String",
                  "localMergedBlocksFetched": "A String",
                  "localMergedBytesRead": "A String",
                  "localMergedChunksFetched": "A String",
                  "mergedFetchFallbackCount": "A String",
                  "remoteMergedBlocksFetched": "A String",
                  "remoteMergedBytesRead": "A String",
                  "remoteMergedChunksFetched": "A String",
                  "remoteMergedReqsDuration": "A String",
                },
              },
              "shuffleWriteMetrics": { # Shuffle data written by task.
                "bytesWritten": "A String",
                "recordsWritten": "A String",
                "writeTimeNanos": "A String",
              },
            },
          },
        },
      ],
    }

      x__xgafv: string, V1 error format.
        Allowed values
          1 - v1 error format
          2 - v2 error format

    Returns:
      An object of the form:

        { # Response returned as an acknowledgement of receipt of data.
    }