A.  

Statistics in the Delta Log will be used to identify partitions that might Include files in the filtered range.

B.  

No file skipping will occur because the optimizer does not know the relationship between the partition column and the longitude.

C.  

The Delta Engine will use row-level statistics in the transaction log to identify the flies that meet the filter criteria.

D.  

Statistics in the Delta Log will be used to identify data files that might include records in the filtered range.

E.  

The Delta Engine will scan the parquet file footers to identify each row that meets the filter criteria.

A.  

spark.sql.files.maxPartitionBytes

B.  

spark.sql.autoBroadcastJoinThreshold

C.  

spark.sql.files.openCostInBytes

D.  

spark.sql.adaptive.coalescePartitions.minPartitionNum

E.  

spark.sql.adaptive.advisoryPartitionSizeInBytes

A.  

Use Repos to merge all differences and make a pull request back to the remote repository.

B.  

Use repos to merge all difference and make a pull request back to the remote repository.

C.  

Use Repos to create a new branch commit all changes and push changes to the remote Git repertory.

D.  

Use repos to create a fork of the remote repository commit all changes and make a pull request on the source repository

A.  

All logic will execute at query time and return the result of joining the valid versions of the source tables at the time the query finishes.

B.  

All logic will execute when the table is defined and store the result of joining tables to the DBFS; this stored data will be returned when the table is queried.

C.  

Results will be computed and cached when the table is defined; these cached results will incrementally update as new records are inserted into source tables.

D.  

All logic will execute at query time and return the result of joining the valid versions of the source tables at the time the query began.

E.  

The versions of each source table will be stored in the table transaction log; query results will be saved to DBFS with each query.

A.  

Because the REST API was used for job creation and triggering runs, a Service Principal will be automatically used to identity these events.

B.  

Because User B last configured the jobs, their identity will be associated with both the job creation events and the job run events.

C.  

Because these events are managed separately, User A will have their identity associated with the job creation events and User B will have their identity associated with the job run events.

D.  

Because the REST API was used for job creation and triggering runs, user identity will not be captured in the audit logs.

E.  

Because User A created the jobs, their identity will be associated with both the job creation events and the job run events.

A.  

An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB.

B.  

Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.

C.  

Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.

D.  

Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.

E.  

An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB.

A.  

When a database is being created, make sure that the LOCATION keyword is used.

B.  

When configuring an external data warehouse for all table storage, leverage Databricks for all ELT.

C.  

When data is saved to a table, make sure that a full file path is specified alongside the Delta format.

D.  

When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.

E.  

When the workspace is being configured, make sure that external cloud object storage has been mounted.

A.  

The Tungsten encoding used by Databricks is optimized for storing string data; newly-added native support for querying JSON strings means that string types are always most efficient.

B.  

Because Delta Lake uses Parquet for data storage, data types can be easily evolved by just modifying file footer information in place.

C.  

Human labor in writing code is the largest cost associated with data engineering workloads; as such, automating table declaration logic should be a priority in all migration workloads.

D.  

Because Databricks will infer schema using types that allow all observed data to be processed, setting types manually provides greater assurance of data quality enforcement.

E.  

Schema inference and evolution on .Databricks ensure that inferred types will always accurately match the data types used by downstream systems.

A.  

importlib.resource path

B.  

,sys.path

C.  

os-path

D.  

pypi.path

E.  

pylib.source

A.  

Schedule a jo to execute the pipeline once and hour on a dedicated interactive cluster.

B.  

Schedule a Structured Streaming job with a trigger interval of 60 minutes.

C.  

Schedule a job to execute the pipeline once hour on a new job cluster.

D.  

Configure a job that executes every time new data lands in a given directory.

A.  

%sh triggers a cluster restart to collect and install Git. Most of the latency is related to cluster startup time.

B.  

Instead of cloning, the code should use %sh pip install so that the Python code can get executed in parallel across all nodes in a cluster.

C.  

%sh does not distribute file moving operations; the final line of code should be updated to use %fs instead.

D.  

Python will always execute slower than Scala on Databricks. The run.py script should be refactored to Scala.

E.  

%sh executes shell code on the driver node. The code does not take advantage of the worker nodes or Databricks optimized Spark.

A.  

Use the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command

B.  

Stop the existing pipeline; use the returned settings in a reset command

C.  

Use the alone command to create a copy of an existing pipeline; use the get JSON command to get the pipeline definition; save this to git

D.  

Use list pipelines to get the specs for all pipelines; get the pipeline spec from the return results parse and use this to create a pipeline

A.  

All columns will be displayed normally for those records that have an age greater than 18; records not meeting this condition will be omitted.

B.  

All columns will be displayed normally for those records that have an age greater than 17; records not meeting this condition will be omitted.

C.  

All age values less than 18 will be returned as null values all other columns will be returned with the values in user_ltv.

D.  

All records from all columns will be displayed with the values in user_ltv.

A.  

The customers table is implemented as a Type 3 table; old values are maintained as a new column alongside the current value.

B.  

The customers table is implemented as a Type 2 table; old values are maintained but marked as no longer current and new values are inserted.

C.  

The customers table is implemented as a Type 0 table; all writes are append only with no changes to existing values.

D.  

The customers table is implemented as a Type 1 table; old values are overwritten by new values and no history is maintained.

E.  

The customers table is implemented as a Type 2 table; old values are overwritten and new customers are appended.

A.  

DBFS is a file system protocol that allows users to interact with files stored in object storage using syntax and guarantees similar to Unix file systems.

B.  

By default, both the DBFS root and mounted data sources are only accessible to workspace administrators.

C.  

The DBFS root is the most secure location to store data, because mounted storage volumes must have full public read and write permissions.

D.  

Neither the DBFS root nor mounted storage can be accessed when using %sh in a Databricks notebook.

E.  

The DBFS root stores files in ephemeral block volumes attached to the driver, while mounted directories will always persist saved data to external storage between sessions.

A.  

%python

B.  

% Databricks notebook source

C.  

-- Databricks notebook source

D.  

//Databricks notebook source

A.  

Regex

B.  

Julia

C.  

pyspsark.ml.feature

D.  

Scala Datasets

E.  

C++

A.  

Databricks has autotuned to a smaller target file size to reduce duration of MERGE operations

B.  

Z-order indices calculated on the table are preventing file compaction

C Bloom filler indices calculated on the table are preventing file compaction

C.  

Databricks has autotuned to a smaller target file size based on the overall size of data in the table

D.  

Databricks has autotuned to a smaller target file size based on the amount of data in each partition

A.  

The connection to the external table will fail; the string "redacted" will be printed.

B.  

An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the encoded password will be saved to DBFS.

C.  

An interactive input box will appear in the notebook; if the right password is provided, the connection will succeed and the password will be printed in plain text.

D.  

The connection to the external table will succeed; the string value of password will be printed in plain text.

E.  

The connection to the external table will succeed; the string "redacted" will be printed.

A.  

Run unit tests against non-production data that closely mirrors production

B.  

Define and unit test functions using Files in Repos

C.  

Define units test and functions within the same notebook

D.  

Define and import unit test functions from a separate Databricks notebook

A.  

The Delta log and Structured Streaming checkpoints record the full history of the Kafka producer.

B.  

Delta Lake schema evolution can retroactively calculate the correct value for newly added fields, as long as the data was in the original source.

C.  

Delta Lake automatically checks that all fields present in the source data are included in the ingestion layer.

D.  

Data can never be permanently dropped or deleted from Delta Lake, so data loss is not possible under any circumstance.

E.  

Ingestine all raw data and metadata from Kafka to a bronze Delta table creates a permanent, replayable history of the data state.

A.  

Model, predict (df, columns)

B.  

Df, map (lambda k:midel (x [columns]) ,select (''customer_id predictions'')

C.  

Df. Select (''customer_id''.

Model (''columns) alias (''predictions'')

D.  

Df.apply(model, columns). Select (''customer_id, prediction''

A.  

Databricks jobs must have exactly one owner; "Owner" privileges cannot be assigned to a group.

B.  

The creator of a Databricks job will always have "Owner" privileges; this configuration cannot be changed.

C.  

Other than the default "admins" group, only individual users can be granted privileges on jobs.

D.  

A user can only transfer job ownership to a group if they are also a member of that group.

E.  

Only workspace administrators can grant "Owner" privileges to a group.

A.  

It closely simulates real world usage of your application.

B.  

It pinpoint errors in the building blocks of your application.

C.  

It provides testing coverage for all code paths and branches.

D.  

It makes it easier to automate your test suite

A.  

Databricks runs HDFS on cloud volume storage; as such, cloud virtual machines must be deployed in the region where the data is stored.

B.  

Databricks workspaces do not rely on any regional infrastructure; as such, the decision should be made based upon what is most convenient for the workspace administrator.

C.  

Cross-region reads and writes can incur significant costs and latency; whenever possible, compute should be deployed in the same region the data is stored.

D.  

Databricks leverages user workstations as the driver during interactive development; as such, users should always use a workspace deployed in a region they are physically near.

E.  

Databricks notebooks send all executable code from the user's browser to virtual machines over the open internet; whenever possible, choosing a workspace region near the end users is the most secure.

A.  

Improves the quality of your data

B.  

Validates a complete use case of your application

C.  

Troubleshooting is easier since all steps are isolated and tested individually

D.  

Yields faster deployment and execution times

E.  

Ensures that all steps interact correctly to achieve the desired end result

A.  

A shuffle occurs prior to writing to try to group data together resulting in fewer files instead of each executor writing multiple files based on directory partitions.

B.  

Optimized writes logical partitions instead of directory partitions partition boundaries are only represented in metadata fewer small files are written.

C.  

An asynchronous job runs after the write completes to detect if files could be further compacted; yes, an OPTIMIZE job is executed toward a default of 1 GB.

D.  

Before a job cluster terminates, OPTIMIZE is executed on all tables modified during the most recent job.

A.  

Each microbatch of a stream-static join will use the most recent version of the static Delta table as of each microbatch.

B.  

Each microbatch of a stream-static join will use the most recent version of the static Delta table as of the job's initialization.

C.  

The checkpoint directory will be used to track state information for the unique keys present in the join.

D.  

Stream-static joins cannot use static Delta tables because of consistency issues.

E.  

The checkpoint directory will be used to track updates to the static Delta table.

A.  

Increase the shuffle partitions to account for additional aggregates

B.  

Specify a new checkpointlocation

C.  

Run REFRESH TABLE delta, /item_agg'

D.  

Remove .option (mergeSchema', true') from the streaming write

A.  

Both commands will succeed. Executing show tables will show that countries at and sales at have been registered as views.

B.  

Cmd 1 will succeed. Cmd 2 will search all accessible databases for a table or view named countries af: if this entity exists, Cmd 2 will succeed.

C.  

Cmd 1 will succeed and Cmd 2 will fail, countries at will be a Python variable representing a PySpark DataFrame.

D.  

Both commands will fail. No new variables, tables, or views will be created.

E.  

Cmd 1 will succeed and Cmd 2 will fail, countries at will be a Python variable containing a list of strings.

A.  

Workspace audit logs

B.  

Driver's log file

C.  

Ganglia

D.  

Cluster Event Log

E.  

Executor's log file

A.  

Integration

B.  

Unit

C.  

Manual

D.  

functional

A.  

Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and overwrite the store_sales_summary table with each Update.

B.  

Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and append new rows nightly to the store_sales_summary table.

C.  

Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.

D.  

Implement the appropriate aggregate logic as a Structured Streaming read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.

E.  

Use Structured Streaming to subscribe to the change data feed for daily_store_sales and apply changes to the aggregates in the store_sales_summary table with each update.

A.  

Nothing will occur until a COMMIT command is executed.

B.  

The table will be removed from the catalog but the data will remain in storage.

C.  

The table will be removed from the catalog and the data will be deleted.

D.  

An error will occur because Delta Lake prevents the deletion of production data.

E.  

Data will be marked as deleted but still recoverable with Time Travel.

A.  

DESCRIBE EXTENDED dev.pii test

B.  

DESCRIBE DETAIL dev.pii test

C.  

SHOW TBLPROPERTIES dev.pii test

D.  

DESCRIBE HISTORY dev.pii test

E.  

SHOW TABLES dev

A.  

date = spark.conf.get("date")

B.  

input_dict = input()

date= input_dict["date"]

C.  

import sys

date = sys.argv[1]

D.  

date = dbutils.notebooks.getParam("date")

E.  

dbutils.widgets.text("date", "null")

date = dbutils.widgets.get("date")