Prepare Important Exam with Associate-Developer-Apache-Spark Exam Dumps(2022) [Q104-Q123]

4.5/5 - (2 votes)

Prepare Important Exam with Associate-Developer-Apache-Spark Exam Dumps(2022)

Pass Exam Questions Efficiently With Associate-Developer-Apache-Spark Questions

QUESTION 104
The code block displayed below contains one or more errors. The code block should load parquet files at location filePath into a DataFrame, only loading those files that have been modified before
2029-03-20 05:44:46. Spark should enforce a schema according to the schema shown below. Find the error.
Schema:
1.root
2. |– itemId: integer (nullable = true)
3. |– attributes: array (nullable = true)
4. | |– element: string (containsNull = true)
5. |– supplier: string (nullable = true)
Code block:
1.schema = StructType([
2. StructType(“itemId”, IntegerType(), True),
3. StructType(“attributes”, ArrayType(StringType(), True), True),
4. StructType(“supplier”, StringType(), True)
5.])
6.
7.spark.read.options(“modifiedBefore”, “2029-03-20T05:44:46”).schema(schema).load(filePath)

The attributes array is specified incorrectly, Spark cannot identify the file format, and the syntax of the call to Spark’s DataFrameReader is incorrect.

Columns in the schema definition use the wrong object type and the syntax of the call to Spark’s DataFrameReader is incorrect.

The data type of the schema is incompatible with the schema() operator and the modification date threshold is specified incorrectly.

Columns in the schema definition use the wrong object type, the modification date threshold is specified incorrectly, and Spark cannot identify the file format.

Columns in the schema are unable to handle empty values and the modification date threshold is specified incorrectly.

Explanation
Correct code block:
schema = StructType([
StructField(“itemId”, IntegerType(), True),
StructField(“attributes”, ArrayType(StringType(), True), True),
StructField(“supplier”, StringType(), True)
])
spark.read.options(modifiedBefore=”2029-03-20T05:44:46″).schema(schema).parquet(filePath) This question is more difficult than what you would encounter in the exam. In the exam, for this question type, only one error needs to be identified and not “one or multiple” as in the question.
Columns in the schema definition use the wrong object type, the modification date threshold is specified incorrectly, and Spark cannot identify the file format.
Correct! Columns in the schema definition should use the StructField type. Building a schema from pyspark.sql.types, as here using classes like StructType and StructField, is one of multiple ways of expressing a schema in Spark. A StructType always contains a list of StructFields (see documentation linked below). So, nesting StructType and StructType as shown in the question is wrong.
The modification date threshold should be specified by a keyword argument like options(modifiedBefore=”2029-03-20T05:44:46″) and not two consecutive non-keyword arguments as in the original code block (see documentation linked below).
Spark cannot identify the file format correctly, because either it has to be specified by using the DataFrameReader.format(), as an argument to DataFrameReader.load(), or directly by calling, for example, DataFrameReader.parquet().
Columns in the schema are unable to handle empty values and the modification date threshold is specified incorrectly.
No. If StructField would be used for the columns instead of StructType (see above), the third argument specified whether the column is nullable. The original schema shows that columns should be nullable and this is specified correctly by the third argument being True in the schema in the code block.
It is correct, however, that the modification date threshold is specified incorrectly (see above).
The attributes array is specified incorrectly, Spark cannot identify the file format, and the syntax of the call to Spark’s DataFrameReader is incorrect.
Wrong. The attributes array is specified correctly, following the syntax for ArrayType (see linked documentation below). That Spark cannot identify the file format is correct, see correct answer above. In addition, the DataFrameReader is called correctly through the SparkSession spark.
Columns in the schema definition use the wrong object type and the syntax of the call to Spark’s DataFrameReader is incorrect.
Incorrect, the object types in the schema definition are correct and syntax of the call to Spark’s DataFrameReader is correct.
The data type of the schema is incompatible with the schema() operator and the modification date threshold is specified incorrectly.
False. The data type of the schema is StructType and an accepted data type for the DataFrameReader.schema() method. It is correct however that the modification date threshold is specified incorrectly (see correct answer above).

QUESTION 105
Which of the following describes Spark’s way of managing memory?

Spark uses a subset of the reserved system memory.

Storage memory is used for caching partitions derived from DataFrames.

As a general rule for garbage collection, Spark performs better on many small objects than few big objects.

Disabling serialization potentially greatly reduces the memory footprint of a Spark application.

Spark’s memory usage can be divided into three categories: Execution, transaction, and storage.

QUESTION 106
Which of the following describes characteristics of the Spark UI?

Via the Spark UI, workloads can be manually distributed across executors.

Via the Spark UI, stage execution speed can be modified.

The Scheduler tab shows how jobs that are run in parallel by multiple users are distributed across the cluster.

There is a place in the Spark UI that shows the property spark.executor.memory.

Some of the tabs in the Spark UI are named Jobs, Stages, Storage, DAGs, Executors, and SQL.

QUESTION 107
Which of the following code blocks creates a new one-column, two-row DataFrame dfDates with column date of type timestamp?

1.dfDates = spark.createDataFrame([“23/01/2022 11:28:12″,”24/01/2022 10:58:34”], [“date”])
2.dfDates = dfDates.withColumn(“date”, to_timestamp(“dd/MM/yyyy HH:mm:ss”, “date”))

1.dfDates = spark.createDataFrame([(“23/01/2022 11:28:12”,),(“24/01/2022 10:58:34”,)], [“date”])
2.dfDates = dfDates.withColumnRenamed(“date”, to_timestamp(“date”, “yyyy-MM-dd HH:mm:ss”))

1.dfDates = spark.createDataFrame([(“23/01/2022 11:28:12”,),(“24/01/2022 10:58:34”,)], [“date”])
2.dfDates = dfDates.withColumn(“date”, to_timestamp(“date”, “dd/MM/yyyy HH:mm:ss”))

1.dfDates = spark.createDataFrame([“23/01/2022 11:28:12″,”24/01/2022 10:58:34”], [“date”])
2.dfDates = dfDates.withColumnRenamed(“date”, to_datetime(“date”, “yyyy-MM-dd HH:mm:ss”))

1.dfDates = spark.createDataFrame([(“23/01/2022 11:28:12”,),(“24/01/2022 10:58:34”,)], [“date”])

Explanation
This question is tricky. Two things are important to know here:
First, the syntax for createDataFrame: Here you need a list of tuples, like so: [(1,), (2,)]. To define a tuple in Python, if you just have a single item in it, it is important to put a comma after the item so that Python interprets it as a tuple and not just a normal parenthesis.
Second, you should understand the to_timestamp syntax. You can find out more about it in the documentation linked below.
For good measure, let’s examine in detail why the incorrect options are wrong:
dfDates = spark.createDataFrame([(“23/01/2022 11:28:12”,),(“24/01/2022 10:58:34”,)], [“date”]) This code snippet does everything the question asks for – except that the data type of the date column is a string and not a timestamp. When no schema is specified, Spark sets the string data type as default.
dfDates = spark.createDataFrame([“23/01/2022 11:28:12″,”24/01/2022 10:58:34”], [“date”]) dfDates = dfDates.withColumn(“date”, to_timestamp(“dd/MM/yyyy HH:mm:ss”, “date”)) In the first row of this command, Spark throws the following error: TypeError: Can not infer schema for type:
<class ‘str’>. This is because Spark expects to find row information, but instead finds strings. This is why you need to specify the data as tuples. Fortunately, the Spark documentation (linked below) shows a number of examples for creating DataFrames that should help you get on the right track here.
dfDates = spark.createDataFrame([(“23/01/2022 11:28:12”,),(“24/01/2022 10:58:34”,)], [“date”]) dfDates = dfDates.withColumnRenamed(“date”, to_timestamp(“date”, “yyyy-MM-dd HH:mm:ss”)) The issue with this answer is that the operator withColumnRenamed is used. This operator simply renames a column, but it has no power to modify its actual content. This is why withColumn should be used instead. In addition, the date format yyyy-MM-dd HH:mm:ss does not reflect the format of the actual timestamp: “23/01/2022 11:28:12”.
dfDates = spark.createDataFrame([“23/01/2022 11:28:12″,”24/01/2022 10:58:34”], [“date”]) dfDates = dfDates.withColumnRenamed(“date”, to_datetime(“date”, “yyyy-MM-dd HH:mm:ss”)) Here, withColumnRenamed is used instead of withColumn (see above). In addition, the rows are not expressed correctly – they should be written as tuples, using parentheses. Finally, even the date format is off here (see above).
More info: pyspark.sql.functions.to_timestamp – PySpark 3.1.2 documentation and pyspark.sql.SparkSession.createDataFrame – PySpark 3.1.1 documentation Static notebook | Dynamic notebook: See test 2

QUESTION 108
Which of the following statements about executors is correct?

Executors are launched by the driver.

Executors stop upon application completion by default.

Each node hosts a single executor.

Executors store data in memory only.

An executor can serve multiple applications.

QUESTION 109
Which of the following code blocks returns a copy of DataFrame transactionsDf that only includes columns transactionId, storeId, productId and f?
Sample of DataFrame transactionsDf:
1.+————-+———+—–+——-+———+—-+
2.|transactionId|predError|value|storeId|productId| f|
3.+————-+———+—–+——-+———+—-+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.+————-+———+—–+——-+———+—-+

transactionsDf.drop(col(“value”), col(“predError”))

transactionsDf.drop(“predError”, “value”)

transactionsDf.drop(value, predError)

transactionsDf.drop([“predError”, “value”])

transactionsDf.drop([col(“predError”), col(“value”)])

QUESTION 110
The code block shown below should return a column that indicates through boolean variables whether rows in DataFrame transactionsDf have values greater or equal to 20 and smaller or equal to
30 in column storeId and have the value 2 in column productId. Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__((__2__.__3__) __4__ (__5__))

1. select
2. col(“storeId”)
3. between(20, 30)
4. and
5. col(“productId”)==2

1. where
2. col(“storeId”)
3. geq(20).leq(30)
4. &
5. col(“productId”)==2

1. select
2. “storeId”
3. between(20, 30)
4. &&
5. col(“productId”)==2

1. select
2. col(“storeId”)
3. between(20, 30)
4. &&
5. col(“productId”)=2

1. select
2. col(“storeId”)
3. between(20, 30)
4. &
5. col(“productId”)==2

QUESTION 111
The code block shown below should add column transactionDateForm to DataFrame transactionsDf. The column should express the unix-format timestamps in column transactionDate as string type like Apr 26 (Sunday). Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__(__2__, from_unixtime(__3__, __4__))

1. withColumn
2. “transactionDateForm”
3. “MMM d (EEEE)”
4. “transactionDate”

1. select
2. “transactionDate”
3. “transactionDateForm”
4. “MMM d (EEEE)”

1. withColumn
2. “transactionDateForm”
3. “transactionDate”
4. “MMM d (EEEE)”

1. withColumn
2. “transactionDateForm”
3. “transactionDate”
4. “MM d (EEE)”

1. withColumnRenamed
2. “transactionDate”
3. “transactionDateForm”
4. “MM d (EEE)”

QUESTION 112
Which of the following is a characteristic of the cluster manager?

Each cluster manager works on a single partition of data.

The cluster manager receives input from the driver through the SparkContext.

The cluster manager does not exist in standalone mode.

The cluster manager transforms jobs into DAGs.

In client mode, the cluster manager runs on the edge node.

QUESTION 113
Which of the following code blocks removes all rows in the 6-column DataFrame transactionsDf that have missing data in at least 3 columns?

transactionsDf.dropna(“any”)

transactionsDf.dropna(thresh=4)

transactionsDf.drop.na(“”,2)

transactionsDf.dropna(thresh=2)

transactionsDf.dropna(“”,4)

QUESTION 114
Which of the following code blocks returns a copy of DataFrame itemsDf where the column supplier has been renamed to manufacturer?

itemsDf.withColumn([“supplier”, “manufacturer”])

itemsDf.withColumn(“supplier”).alias(“manufacturer”)

itemsDf.withColumnRenamed(“supplier”, “manufacturer”)

itemsDf.withColumnRenamed(col(“manufacturer”), col(“supplier”))

itemsDf.withColumnsRenamed(“supplier”, “manufacturer”)

Explanation
itemsDf.withColumnRenamed(“supplier”, “manufacturer”)
Correct! This uses the relatively trivial DataFrame method withColumnRenamed for renaming column supplier to column manufacturer.
Note that the question asks for “a copy of DataFrame itemsDf”. This may be confusing if you are not familiar with Spark yet. RDDs (Resilient Distributed Datasets) are the foundation of Spark DataFrames and are immutable. As such, DataFrames are immutable, too. Any command that changes anything in the DataFrame therefore necessarily returns a copy, or a new version, of it that has the changes applied.
itemsDf.withColumnsRenamed(“supplier”, “manufacturer”)
Incorrect. Spark’s DataFrame API does not have a withColumnsRenamed() method.
itemsDf.withColumnRenamed(col(“manufacturer”), col(“supplier”))
No. Watch out – although the col() method works for many methods of the DataFrame API, withColumnRenamed is not one of them. As outlined in the documentation linked below, withColumnRenamed expects strings.
itemsDf.withColumn([“supplier”, “manufacturer”])
Wrong. While DataFrame.withColumn() exists in Spark, it has a different purpose than renaming columns.
withColumn is typically used to add columns to DataFrames, taking the name of the new column as a first, and a Column as a second argument. Learn more via the documentation that is linked below.
itemsDf.withColumn(“supplier”).alias(“manufacturer”)
No. While DataFrame.withColumn() exists, it requires 2 arguments. Furthermore, the alias() method on DataFrames would not help the cause of renaming a column much. DataFrame.alias() can be useful in addressing the input of join statements. However, this is far outside of the scope of this question. If you are curious nevertheless, check out the link below.
More info: pyspark.sql.DataFrame.withColumnRenamed – PySpark 3.1.1 documentation, pyspark.sql.DataFrame.withColumn – PySpark 3.1.1 documentation, and pyspark.sql.DataFrame.alias – PySpark 3.1.2 documentation (https://bit.ly/3aSB5tm , https://bit.ly/2Tv4rbE , https://bit.ly/2RbhBd2) Static notebook | Dynamic notebook: See test 1 (https://flrs.github.io/spark_practice_tests_code/#1/31.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

QUESTION 115
The code block displayed below contains an error. The code block should return a DataFrame where all entries in column supplier contain the letter combination et in this order. Find the error.
Code block:
itemsDf.filter(Column(‘supplier’).isin(‘et’))

The Column operator should be replaced by the col operator and instead of isin, contains should be used.

The expression inside the filter parenthesis is malformed and should be replaced by isin(‘et’, ‘supplier’).

Instead of isin, it should be checked whether column supplier contains the letters et, so isin should be replaced with contains. In addition, the column should be accessed using col[‘supplier’].

The expression only returns a single column and filter should be replaced by select.

QUESTION 116
The code block displayed below contains an error. The code block should return a DataFrame in which column predErrorAdded contains the results of Python function add_2_if_geq_3 as applied to numeric and nullable column predError in DataFrame transactionsDf. Find the error.
Code block:
1.def add_2_if_geq_3(x):
2. if x is None:
3. return x
4. elif x >= 3:
5. return x+2
6. return x
7.
8.add_2_if_geq_3_udf = udf(add_2_if_geq_3)
9.
10.transactionsDf.withColumnRenamed(“predErrorAdded”, add_2_if_geq_3_udf(col(“predError”)))

The operator used to adding the column does not add column predErrorAdded to the DataFrame.

Instead of col(“predError”), the actual DataFrame with the column needs to be passed, like so transactionsDf.predError.

The udf() method does not declare a return type.

UDFs are only available through the SQL API, but not in the Python API as shown in the code block.

The Python function is unable to handle null values, resulting in the code block crashing on execution.

QUESTION 117
Which of the following describes a shuffle?

A shuffle is a process that is executed during a broadcast hash join.

A shuffle is a process that compares data across executors.

A shuffle is a process that compares data across partitions.

A shuffle is a Spark operation that results from DataFrame.coalesce().

A shuffle is a process that allocates partitions to executors.

QUESTION 118
The code block displayed below contains an error. The code block should return a copy of DataFrame transactionsDf where the name of column transactionId has been changed to transactionNumber. Find the error.
Code block:
transactionsDf.withColumn(“transactionNumber”, “transactionId”)

The arguments to the withColumn method need to be reordered.

The arguments to the withColumn method need to be reordered and the copy() operator should be appended to the code block to ensure a copy is returned.

The copy() operator should be appended to the code block to ensure a copy is returned.

Each column name needs to be wrapped in the col() method and method withColumn should be replaced by method withColumnRenamed.

The method withColumn should be replaced by method withColumnRenamed and the arguments to the method need to be reordered.

QUESTION 119
Which of the following describes the role of the cluster manager?

The cluster manager schedules tasks on the cluster in client mode.

The cluster manager schedules tasks on the cluster in local mode.

The cluster manager allocates resources to Spark applications and maintains the executor processes in client mode.

The cluster manager allocates resources to Spark applications and maintains the executor processes in remote mode.

The cluster manager allocates resources to the DataFrame manager.

QUESTION 120
Which of the following code blocks returns about 150 randomly selected rows from the 1000-row DataFrame transactionsDf, assuming that any row can appear more than once in the returned DataFrame?

transactionsDf.resample(0.15, False, 3142)

transactionsDf.sample(0.15, False, 3142)

transactionsDf.sample(0.15)

transactionsDf.sample(0.85, 8429)

transactionsDf.sample(True, 0.15, 8261)

QUESTION 121
Which of the following describes Spark actions?

Writing data to disk is the primary purpose of actions.

Actions are Spark’s way of exchanging data between executors.

The driver receives data upon request by actions.

Stage boundaries are commonly established by actions.

Actions are Spark’s way of modifying RDDs.

QUESTION 122
The code block shown below should return a DataFrame with all columns of DataFrame transactionsDf, but only maximum 2 rows in which column productId has at least the value 2. Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__(__2__).__3__

1. where
2. “productId” > 2
3. max(2)

1. where
2. transactionsDf[productId] >= 2
3. limit(2)

1. filter
2. productId > 2
3. max(2)

1. filter
2. col(“productId”) >= 2
3. limit(2)

1. where
2. productId >= 2
3. limit(2)

QUESTION 123
Which of the following code blocks silently writes DataFrame itemsDf in avro format to location fileLocation if a file does not yet exist at that location?

itemsDf.write.avro(fileLocation)

itemsDf.write.format(“avro”).mode(“ignore”).save(fileLocation)

itemsDf.write.format(“avro”).mode(“errorifexists”).save(fileLocation)

itemsDf.save.format(“avro”).mode(“ignore”).write(fileLocation)

spark.DataFrameWriter(itemsDf).format(“avro”).write(fileLocation)

New Way To Prepare For Your Databricks Associate Developer Apache Spark Exam

Is Databricks Associate Developer Apache Spark Exam worth it?

Read Our Databricks Associate Developer Apache Spark Exam Guide That Will Help You Pass in First Try

Do you want to know everything you need to know in order to pass the Databricks Associate Developer Apache Spark exam in less than an hour?

It’s always a good idea to know what the best material for preparing yourself for an exam is. In fact, it’s a requirement. When it comes to the Apache Spark exam, however, many people don’t really know where to begin in order to prepare themselves. They get stuck feeling overwhelmed and confused.

If you’re like most people out there who have been struggling with taking the exam, you’ve probably been feeling like you have no idea what to do. And the more time you spend trying to figure this out, the longer it will take you to pass the test.

In this article, I’m going to tell you about how I personally used to prepare myself for this test and how you too can do the same. Databricks Associate Developer Apache Spark exam dumps will help you pass your exam in first try.

So if you want to know everything you need to know in order to pass the Databricks Associate Developer Apache Spark exam in less than an hour, then read on&mldr;

Associate-Developer-Apache-Spark Questions – Truly Beneficial For Your Databricks Exam: https://www.testsimulate.com/Associate-Developer-Apache-Spark-study-materials.html

New Way To Prepare For Your Databricks Associate Developer Apache Spark Exam

Leave a Reply Cancel reply