Nitro Workspace SDK
To avoid performance issues when moving very large data sets, data scientists can now use the Nitro Workspace SDK to gain full access to their personal workspace data lake location. They can then write their analyzed result sets back to Nitro workspaces directly into the Nitro data lake instead of the data warehouse.
The Nitro Workspace SDK provides the following functionality:
- Validates Nitro user for workspace access
- Grants the user all necessary permissions to perform S3 operations for the duration of the session
- Supported file operations include csv, parquet, and json
- Available operations:
- List directories/files under exact workspace location in S3
- Delete file from exact workspace location in S3
- Upload files from local file system to exact workspace location in S3
- Download files from exact workspace location in S3 to local filesystem
- Read files from exact workspace location in S3 into Pandas dataframe
- Write Pandas dataframe as a file to exact workspace location in S3
- Unload Redshift table to files in exact workspace location in S3
- Copy files in exact workspace location in S3 into Redshift table
- Python 3.6 - Python 3.9 installed on your machine
- Named NAC user with dev admin permissions for the Tenant
To use the Nitro Workspace SDK (Beta) for Mac:
Go to terminal and type vi.cdw under the home directory
For each tenant instance that you have a workspace(s) in, enter the following information:
If you don’t have an API key for the tenant instance NAC, generate one before filling in the above information.
Download the installable wheel file (for example, nitro_sdk-1.0.0.py3-none-any.whl) to your local computer.
Open a Command Prompt and run the following commands, in this order:Copy
# create new venv
python3 -m venv /path/to/new/virtual/environment
# Activate new venv
# pip install the wheel file (installs in venv with all the dependencies)
pip3 install /path/to/wheel/nitro_sdk-1.0.0-py3-none-any.whl
# pip force-re-install the wheel file (if you are upgrading the version)
pip3 install --force-reinstall nitro_sdk-1.0.0-py3-none-any.whl
- Open a Python IDE in the same virtual environment to begin writing code. Access the jupyter notebook template or readme.md file for more information on the methods.
To use jupyter notebook, run the following command in your terminal:
python3 -m pip install jupyter notebook
python3 -m ipykernel install --user --name=virtualenvname (edit this)
Install the packages you need into your active virtual environment.
Using Nitro SDK video - pre-requisites, installing, and examples