Phoenix Data Tools

Welcome the phoenixdatatools library

PhoenixDataTools is a Python library specifically designed to optimize development in Databricks environments using PySpark. This library is ideal for integration into Databricks notebooks and jobs, offering a simplified and efficient interface for data manipulation.

The primary goal of PhoenixDataTools is to provide an abstraction layer that simplifies and accelerates working with small, medium, and large volumes of data, eliminating the need to write extensive lines of repetitive code. The library facilitates complex operations with DataFrames, Delta table manipulation, and management of storage and reading connections. This results in a significant improvement in productivity and code maintenance.

Install

The library is published directly to Pipy, which is a de facto standard package management system used to install and manage software packages written in Python.

pip install phoenixdatatools

After you import and utilize:

from phoenixdatatools import data_quality

data_quality.not_null_test(df, columns)

Documentation

For more details about each module and function, see the following sections:

• Data Quality

• Data Analysis

• Data Security

• Data Fake

Contributing and Support

If you would like to contribute to phoenixdatatools, please review our contribution guidelines.

If you encounter any issues or have any questions, feel free to open an issue in our Bitbucket repository.

License

Phoenix Data Tools is licensed under the MIT License.