Databricks Certified-Data-Engineer-Professional 시험 개요:

Databricks Certified-Data-Engineer-Professional 시험 요강 주제:

최신 Databricks Certification Certified-Data-Engineer-Professional 무료샘플문제

문제 #1

Which statement regarding stream-static joins and static Delta tables is correct?

A. The checkpoint directory will be used to track updates to the static Delta table.
B. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of each microbatch.
C. Stream-static joins cannot use static Delta tables because of consistency issues.
D. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of the job's initialization.
E. The checkpoint directory will be used to track state information for the unique keys present in the join.

문제 #2

A data engineer is reviewing the PySpark code to copy a part of the production dataset to the sandbox environment, and needs to be sure that no PII(Personally Identifiable Information) data is being copied. After checking the sales table, the data engineer notices that it has user emails as the only PII data included as well as being the only column to identify the user.
from pyspark.sql import functions as F

Which anonymised code should be used to achieve the required outcome?

A. df.withColumn ("user_emai", F.expr("uuid()"))
B. df.withColumn ("user_email", F.sha2 ("user_email"))
C. df.withColumn ("user_email", F.regexp_replace ("user_eamail", "@*", "@anonymized.com"))
D. df.withColumn ("hashed_email", sha2 ("user_email"))

문제 #3

A junior member of the data engineering team is exploring the language interoperability of Databricks notebooks. The intended outcome of the below code is to register a view of all sales that occurred in countries on the continent of Africa that appear in the geo_lookup table.
Before executing the code, running SHOW TABLES on the current database indicates the database contains only two tables: geo_lookup and sales.

Which statement correctly describes the outcome of executing these command cells in order in an interactive notebook?

A. Both commands will fail. No new variables, tables, or views will be created.
B. Cmd 1 will succeed and Cmd 2 will fail, countries at will be a Python variable representing a PySpark DataFrame.
C. Cmd 1 will succeed and Cmd 2 will fail, countries at will be a Python variable containing a list of strings.
D. Cmd 1 will succeed. Cmd 2 will search all accessible databases for a table or view named countries af: if this entity exists, Cmd 2 will succeed.
E. Both commands will succeed. Executing show tables will show that countries at and sales at have been registered as views.

문제 #4

A data engineer is using Structured Streaming to read in transaction data from a bronze Delta table. It was discovered that the data has quality issues where sometimes the transaction value is negative, and when that occurs, the rows need to be routed to a separate quarantine table. They have low latency requirements for the good data since it is used by downstream systems, but the bad data will only be analyzed periodically and has no production dependencies. The quarantine job needs to be implemented so that it cannot affect the production processes that depend on the good data, and the cost of the job needs to be minimized. How should the quarantine process be implemented in order to satisfy these requirements?

A. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing, and should not share compute with other processes. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing, and should be implemented on a separate small cluster and only run once a day to minimize cost.
B. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. A new boolean column called "quarantine" should be added to the dataframe, and its value should be set to true if the transaction value is less than 0 and false if the transaction value is greater than or equal to 0. Processing and storing all the data together will save costs.
C. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. Inside a foreachBatch function, the dataframe should be filtered so that records with a transaction value greater than or equal to 0 are written to the good data table and records with a transaction value less than 0 are written to a quarantine table. Try/Catch can be added around the writes in the foreachBatch function so that the stream can't fail.
D. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing. Both should run as separate streams on the same cluster to minimize cost.

문제 #5

A data team is working to optimize an existing large, fast-growing table 'orders' with high cardinality columns, which experiences significant data skew and requires frequent concurrent writes. The team notice that the columns 'user_id', 'event_timestamp' and 'product_id' are heavily used in analytical queries and filters, although those keys may be subject to change in the future due to different business requirements. Which partitioning strategy should the team choose to optimize the table for immediate data skipping, incremental management over time, and flexibility?

A. Z-order the table with OPTIMIZE orders ZORDER BY (user_id, product_id, event_timestamp)
B. Partition the table with: ALTER TABLE orders PARTITION BY user_id, product_id, event_timestamp
C. Cluster the table with: ALTER TABLE orders CLUSTER BY user_id, product_id, event_timestamp
D. Use z-order after partitiing the table: OPTIMIZE orders ZORDER BY (user_id, product_id) WHERE event_timestamp = current date () - 1 DAY

질문과 대답: