Decrypts specified columns in a DataFrame using the provided key.

Parameters:
  • data_frame (DataFrame) –

    Input DataFrame with data to be decrypted.

  • columns_to_decrypt (list) –

    List of column names to decrypt.

  • key (bytes) –

    Decryption key to use for decryption.
Returns:
  • DataFrame( DataFrame ) –

    A DataFrame with decrypted columns.
Raises:
  • Exception –

    If any of the columns to decrypt are not found in the DataFrame.

Examples:

>>> from pyspark.sql import SparkSession
>>> from pyspark.sql.types import StructType, StructField, StringType
>>> spark = SparkSession.builder.getOrCreate()
>>> schema = StructType([
...     StructField("id", StringType(), True),
...     StructField("name", StringType(), True)
... ])
>>> data = [("1", "gAAAAABg0f8JqXc2..."), ("2", "gAAAAABg0f8JqXc2...")]
>>> df = spark.createDataFrame(data, schema)
>>> key = generate_encryption_key()
>>> decrypted_df = decrypt_dataframe(df, ["name"], key)

Logs:

  • ERROR: Decryption Failed: column is not present in the DataFrame!

  • INFO: DataFrame columns columns_to_decrypt decrypted successfully.