Associate-Developer-Apache-Spark-3.5最も有効な質問と解答で勉強

A data scientist is working on a project that requires processing large amounts of structured data, performing SQL queries, and applying machine learning algorithms. The data scientist is considering using Apache Spark for this task.
Which combination of Apache Spark modules should the data scientist use in this scenario?
Options:

• A. Spark DataFrames, Spark SQL, and MLlib
• B. Spark SQL, Pandas API on Spark, and Structured Streaming
• C. Spark Streaming, GraphX, and Pandas API on Spark
• D. Spark DataFrames, Structured Streaming, and GraphX

A data engineer is working on a Streaming DataFrame streaming_df with the given streaming data:

Which operation is supported with streamingdf ?

• A. streaming_df.groupby("Id") .count ()
• B. streaming_df.filter (col("count") < 30).show()
• C. streaming_df.orderBy("timestamp").limit(4)
• D. streaming_df. select (countDistinct ("Name") )

A data analyst wants to add a column date derived from a timestamp column.
Options:

• A. dates_df.withColumn("date", f.date_format("timestamp", "yyyy-MM-dd")).show()
• B. dates_df.withColumn("date", f.to_date("timestamp")).show()
• C. dates_df.withColumn("date", f.unix_timestamp("timestamp")).show()
• D. dates_df.withColumn("date", f.from_unixtime("timestamp")).show()

38 of 55.
A data engineer is working with Spark SQL and has a large JSON file stored at /data/input.json.
The file contains records with varying schemas, and the engineer wants to create an external table in Spark SQL that:
Reads directly from /data/input.json.
Infers the schema automatically.
Merges differing schemas.
Which code snippet should the engineer use?

• A. CREATE EXTERNAL TABLE users
  USING json
  OPTIONS (path '/data/input.json', mergeSchema 'true');
• B. CREATE EXTERNAL TABLE users
  USING json
  OPTIONS (path '/data/input.json', mergeAll 'true');
• C. CREATE TABLE users
  USING json
  OPTIONS (path '/data/input.json');
• D. CREATE EXTERNAL TABLE users
  USING json
  OPTIONS (path '/data/input.json', inferSchema 'true');

23 of 55.
A data scientist is working with a massive dataset that exceeds the memory capacity of a single machine. The data scientist is considering using Apache Spark™ instead of traditional single-machine languages like standard Python scripts.
Which two advantages does Apache Spark™ offer over a normal single-machine language in this scenario? (Choose 2 answers)

• A. It eliminates the need to write any code, automatically handling all data processing.
• B. It processes data solely on disk storage, reducing the need for memory resources.
• C. It requires specialized hardware to run, making it unsuitable for commodity hardware clusters.
• D. It has built-in fault tolerance, allowing it to recover seamlessly from node failures during computation.
• E. It can distribute data processing tasks across a cluster of machines, enabling horizontal scalability.