Pass Associate-Developer-Apache-Spark Brain Dump Updated Certification Sample Questions [Q11-Q30]

NEW QUESTION 11
The code block displayed below contains an error. The code block should combine data from DataFrames itemsDf and transactionsDf, showing all rows of DataFrame itemsDf that have a matching value in column itemId with a value in column transactionsId of DataFrame transactionsDf. Find the error.
Code block:
itemsDf.join(itemsDf.itemId==transactionsDf.transactionId)

The join statement is incomplete.

The union method should be used instead of join.

The join method is inappropriate.

The merge method should be used instead of join.

The join expression is malformed.

NEW QUESTION 12
Which of the following is a characteristic of the cluster manager?

Each cluster manager works on a single partition of data.

The cluster manager receives input from the driver through the SparkContext.

The cluster manager does not exist in standalone mode.

The cluster manager transforms jobs into DAGs.

In client mode, the cluster manager runs on the edge node.

NEW QUESTION 13
Which of the following code blocks returns a one-column DataFrame for which every row contains an array of all integer numbers from 0 up to and including the number given in column predError of DataFrame transactionsDf, and null if predError is null?
Sample of DataFrame transactionsDf:
1.+————-+———+—–+——-+———+—-+
2.|transactionId|predError|value|storeId|productId| f|
3.+————-+———+—–+——-+———+—-+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.| 4| null| null| 3| 2|null|
8.| 5| null| null| null| 2|null|
9.| 6| 3| 2| 25| 2|null|
10.+————-+———+—–+——-+———+—-+

1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = [range(target)]
6. return result
7.
8.count_to_target_udf = udf(count_to_target, ArrayType[IntegerType])
9.
10.transactionsDf.select(count_to_target_udf(col(‘predError’)))

1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = list(range(target))
6. return result
7.
8.transactionsDf.select(count_to_target(col(‘predError’)))

1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = list(range(target))
6. return result
7.
8.count_to_target_udf = udf(count_to_target, ArrayType(IntegerType()))
9.
10.transactionsDf.select(count_to_target_udf(‘predError’))
(Correct)

1.def count_to_target(target):
2. result = list(range(target))
3. return result
4.
5.count_to_target_udf = udf(count_to_target, ArrayType(IntegerType()))
6.
7.df = transactionsDf.select(count_to_target_udf(‘predError’))

1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = list(range(target))
6. return result
7.
8.count_to_target_udf = udf(count_to_target)
9.
10.transactionsDf.select(count_to_target_udf(‘predError’))

NEW QUESTION 14
Which of the following code blocks silently writes DataFrame itemsDf in avro format to location fileLocation if a file does not yet exist at that location?

itemsDf.write.avro(fileLocation)

itemsDf.write.format(“avro”).mode(“ignore”).save(fileLocation)

itemsDf.write.format(“avro”).mode(“errorifexists”).save(fileLocation)

itemsDf.save.format(“avro”).mode(“ignore”).write(fileLocation)

spark.DataFrameWriter(itemsDf).format(“avro”).write(fileLocation)

NEW QUESTION 15
The code block shown below should return a DataFrame with columns transactionsId, predError, value, and f from DataFrame transactionsDf. Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__(__2__)

1. filter
2. “transactionId”, “predError”, “value”, “f”

1. select
2. “transactionId, predError, value, f”

1. select
2. [“transactionId”, “predError”, “value”, “f”]

1. where
2. col(“transactionId”), col(“predError”), col(“value”), col(“f”)

1. select
2. col([“transactionId”, “predError”, “value”, “f”])

NEW QUESTION 16
The code block displayed below contains an error. The code block should return all rows of DataFrame transactionsDf, but including only columns storeId and predError. Find the error.
Code block:
spark.collect(transactionsDf.select(“storeId”, “predError”))

Instead of select, DataFrame transactionsDf needs to be filtered using the filter operator.

Columns storeId and predError need to be represented as a Python list, so they need to be wrapped in brackets ([]).

The take method should be used instead of the collect method.

Instead of collect, collectAsRows needs to be called.

The collect method is not a method of the SparkSession object.

NEW QUESTION 17
The code block shown below should return a DataFrame with all columns of DataFrame transactionsDf, but only maximum 2 rows in which column productId has at least the value 2. Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__(__2__).__3__

1. where
2. “productId” > 2
3. max(2)

1. where
2. transactionsDf[productId] >= 2
3. limit(2)

1. filter
2. productId > 2
3. max(2)

1. filter
2. col(“productId”) >= 2
3. limit(2)

1. where
2. productId >= 2
3. limit(2)

NEW QUESTION 18
Which of the following code blocks writes DataFrame itemsDf to disk at storage location filePath, making sure to substitute any existing data at that location?

itemsDf.write.mode(“overwrite”).parquet(filePath)

itemsDf.write.option(“parquet”).mode(“overwrite”).path(filePath)

itemsDf.write(filePath, mode=”overwrite”)

itemsDf.write.mode(“overwrite”).path(filePath)

itemsDf.write().parquet(filePath, mode=”overwrite”)

NEW QUESTION 19
Which of the following describes Spark actions?

Writing data to disk is the primary purpose of actions.

Actions are Spark’s way of exchanging data between executors.

The driver receives data upon request by actions.

Stage boundaries are commonly established by actions.

Actions are Spark’s way of modifying RDDs.

NEW QUESTION 20
Which of the following code blocks immediately removes the previously cached DataFrame transactionsDf from memory and disk?

array_remove(transactionsDf, “*”)

transactionsDf.unpersist()
(Correct)

del transactionsDf

transactionsDf.clearCache()

transactionsDf.persist()

NEW QUESTION 21
Which of the following code blocks returns a new DataFrame with the same columns as DataFrame transactionsDf, except for columns predError and value which should be removed?

transactionsDf.drop([“predError”, “value”])

transactionsDf.drop(“predError”, “value”)

transactionsDf.drop(col(“predError”), col(“value”))

transactionsDf.drop(predError, value)

transactionsDf.drop(“predError & value”)

NEW QUESTION 22
Which of the following code blocks displays the 10 rows with the smallest values of column value in DataFrame transactionsDf in a nicely formatted way?

transactionsDf.sort(asc(value)).show(10)

transactionsDf.sort(col(“value”)).show(10)

transactionsDf.sort(col(“value”).desc()).head()

transactionsDf.sort(col(“value”).asc()).print(10)

transactionsDf.orderBy(“value”).asc().show(10)

NEW QUESTION 23
Which of the following code blocks reorders the values inside the arrays in column attributes of DataFrame itemsDf from last to first one in the alphabet?
1.+——+—————————–+——————-+
2.|itemId|attributes |supplier |
3.+——+—————————–+——————-+
4.|1 |[blue, winter, cozy] |Sports Company Inc.|
5.|2 |[red, summer, fresh, cooling]|YetiX |
6.|3 |[green, summer, travel] |Sports Company Inc.|
7.+——+—————————–+——————-+

itemsDf.withColumn(‘attributes’, sort_array(col(‘attributes’).desc()))

itemsDf.withColumn(‘attributes’, sort_array(desc(‘attributes’)))

itemsDf.withColumn(‘attributes’, sort(col(‘attributes’), asc=False))

itemsDf.withColumn(“attributes”, sort_array(“attributes”, asc=False))

itemsDf.select(sort_array(“attributes”))

NEW QUESTION 24
The code block displayed below contains an error. The code block is intended to write DataFrame transactionsDf to disk as a parquet file in location /FileStore/transactions_split, using column storeId as key for partitioning. Find the error.
Code block:
transactionsDf.write.format(“parquet”).partitionOn(“storeId”).save(“/FileStore/transactions_split”)A.

The format(“parquet”) expression is inappropriate to use here, “parquet” should be passed as first argument to the save() operator and “/FileStore/transactions_split” as the second argument.

Partitioning data by storeId is possible with the partitionBy expression, so partitionOn should be replaced by partitionBy.

Partitioning data by storeId is possible with the bucketBy expression, so partitionOn should be replaced by bucketBy.

partitionOn(“storeId”) should be called before the write operation.

The format(“parquet”) expression should be removed and instead, the information should be added to the write expression like so: write(“parquet”).

NEW QUESTION 25
Which of the following code blocks returns a DataFrame where columns predError and productId are removed from DataFrame transactionsDf?
Sample of DataFrame transactionsDf:
1.+————-+———+—–+——-+———+—-+
2.|transactionId|predError|value|storeId|productId|f |
3.+————-+———+—–+——-+———+—-+
4.|1 |3 |4 |25 |1 |null|
5.|2 |6 |7 |2 |2 |null|
6.|3 |3 |null |25 |3 |null|
7.+————-+———+—–+——-+———+—-+

transactionsDf.withColumnRemoved(“predError”, “productId”)

transactionsDf.drop([“predError”, “productId”, “associateId”])

transactionsDf.drop(“predError”, “productId”, “associateId”)

transactionsDf.dropColumns(“predError”, “productId”, “associateId”)

transactionsDf.drop(col(“predError”, “productId”))

NEW QUESTION 26
Which of the following code blocks generally causes a great amount of network traffic?

DataFrame.select()

DataFrame.coalesce()

DataFrame.collect()

DataFrame.rdd.map()

DataFrame.count()

NEW QUESTION 27
The code block shown below should read all files with the file ending .png in directory path into Spark.
Choose the answer that correctly fills the blanks in the code block to accomplish this.
spark.__1__.__2__(__3__).option(__4__, “*.png”).__5__(path)

1. read()
2. format
3. “binaryFile”
4. “recursiveFileLookup”
5. load

1. read
2. format
3. “binaryFile”
4. “pathGlobFilter”
5. load

1. read
2. format
3. binaryFile
4. pathGlobFilter
5. load

1. open
2. format
3. “image”
4. “fileType”
5. open

1. open
2. as
3. “binaryFile”
4. “pathGlobFilter”
5. load

NEW QUESTION 28
Which of the following statements about RDDs is incorrect?

An RDD consists of a single partition.

The high-level DataFrame API is built on top of the low-level RDD API.

RDDs are immutable.

RDD stands for Resilient Distributed Dataset.

RDDs are great for precisely instructing Spark on how to do a query.

NEW QUESTION 29
The code block displayed below contains an error. The code block should configure Spark so that DataFrames up to a size of 20 MB will be broadcast to all worker nodes when performing a join.
Find the error.
Code block:

spark.conf.set(“spark.sql.autoBroadcastJoinThreshold”, 20)

Spark will only broadcast DataFrames that are much smaller than the default value.

The correct option to write configurations is through spark.config and not spark.conf.

Spark will only apply the limit to threshold joins and not to other joins.

The passed limit has the wrong variable type.

The command is evaluated lazily and needs to be followed by an action.

NEW QUESTION 30
Which of the following code blocks returns a DataFrame showing the mean value of column “value” of DataFrame transactionsDf, grouped by its column storeId?

transactionsDf.groupBy(col(storeId).avg())

transactionsDf.groupBy(“storeId”).avg(col(“value”))

transactionsDf.groupBy(“storeId”).agg(avg(“value”))

transactionsDf.groupBy(“storeId”).agg(average(“value”))

transactionsDf.groupBy(“value”).average()

Leave a Reply Cancel reply