Loading Data Into File-Based Connectors Using Table Object Configurations

To eliminate the need for admins to configure job orchestrations or trigger files, users can now run a single, pre-packaged job. This job processes single or multiple table loads based on table object configurations for the Symphony APLD Claims, Symphony PTD Claims industry connectors, and all custom connectors. This includes optionally pulling the data from remote S3 sources if the Enable S3 Pull check box is selected.

To create a new inbound connector using table object configurations:

  1. Log into the NAC.
  2. Select Connectors > Inbound Connectors from the side menu.
  3. Select the New Connector button.
  4. Add connector details including display name and description.

  5. Select a File Based Connector Type.

    Only Symphony APLD Claims and Symphony PTD Claims and Custom are supported at this time.

  6. Select the Enable S3 Pull check box. This step is optional.
  7. Select Save. Once the connector is saved, the table object configurations and connector schemas are automatically added.

Symphony APLD and Symphony PTD connectors have pre-packaged table object configurations. For custom connectors, these configurations are self-service and can be added from the NAC as part of the table object configurations.

Considerations

  • Before adding the table object configurations, admins must ensure the custom connector is created and the metadata is uploaded. Object schema loading patterns and other file properties remain in the table YAMLs
  • File based connectors using the traditional FTP intelligent load job continue to work

Adding Table Object Configurations

To add Table Object Configurations:

  1. Select the Edit button.

  2. Select the Edit icon next to the configuration you want to update. The following attributes are set in the Table Object Configuration:

    • Name – Identifies the table object configuration in Nitro. Must be unique from other table object configurations and tables in the connector, otherwise table ymls cannot be updated.
    • Target Table – The table in Nitro where the data matching these criteria should be loaded
    • Source Filename Pattern – The filename to be loaded into the Target Table. Wildcards and regex are supported.
    • Load Batch Size – The number of files loaded at a time. Is set to “1” when loading into Redshift tables and “-1” when loading into the data lake. This field cannot be edited. For values other than -1, multi-file load check box should be selected.
    • MultiFile Load – Select if this Table Object Configuration should look for more than one file when loading files
    • Active – Select if the Table Object Configuration should be run when running the job “FTP Connector Load”
    • Add Validation – Add data quality rules or rule groups. If violations exceed the threshold, data is not loaded into ODS. This is optional.

  3. Select Save.