ConSol allows for an automated integration to reduce the manual process of importing documents in bulk.
Documents will be picked up for processing from a named S3 bucket. For the automated (bulk) process, a zip file of relevant documents for each order or project view should be delivered to this site. Each zip should be named for the target object it will be uploaded to.
Document Import zip files can be manually uploaded in ConSol but this will be done on an Order by Order basis. Bulk imports for multiple target objects will need to follow the below process.
The system requirements and process are as follows:
- Files will be delivered to the correct customer-specific folder in the yarris-consol-interfaces-prod S3 bucket.
- Separate sub-folders will exist for each target object type (ie Project Views and Orders).
- All documents for each instance of the target object will be zipped into a single file which will be named for the target object that the zip will be uploaded to.
- Project Views will be named using the unique Project View Id
- Orders will be named using the unique Customer Ref
- ConSol will poll this site every minute and will attempt to upload files that are found.
- If a target object corresponding to the zip file name is unable to be found in ConSol, after an hour the zip will be moved to the error folder.
- Files that are larger than 1Gb will be automatically moved to the error folder.
NOTE:
The Error bucket in S3 should be regularly checked for zip files that failed to process. Failure can only occur for one of two reasons:
o The zip file was too large (over 1 Gb)
o The target entity could not be found. Either the target entity had not yet been created or the zip file was incorrectly named. Zips can be moved back from the error folder to the processing folder once the reason for failure has been addressed.
- Each zip file will contain all documents to be uploaded to the target object it is named for. In addition, the following optional target-specific System files can created:
- CONTROL File (eg B9999999_CONTROL.csv) – this file will be in CSV format with two columns – ‘User Id’ and “Leave As Zip’.
- User Id , mandatory if a CONTROL file is used, must be a valid active ConSol user. The user driving the import, who may be different to the owner of the project view or order This user should have the correct ConSol profile (role and workgroup membership) to access the project view or order that the documents are to be loaded against.
- Leave As Zip, if required, contains either TRUE or FALSE – if not specified, the zip will be expanded when processed. Whether the import file will be attached as a single zip or the documents within the zip will be extracted and individually loaded, as part of the import process.
- MANIFEST File (eg B9999999_MANIFEST.csv) – this file allows extra metadata to be specified for each document (keyed to the document name) that is to be uploaded to a target entity. The first column of this file will be the Document Name. The following optional metadata can also be specified:
- Document Type – a valid Document Type that has been defined in ConSol
- Document Description – a text description of the document
- View By – either CUST (Customer Only) or SUPP (Both Customer and Supplier)
- Original Document Path – where the document was originally stored
- Original Document Reference – any additional document reference
NOTE:
The first row in the MANIFEST file should leave the Document Name blank, in which case the values stored in the Metadata columns will used as defaults for all documents within the zip. Documents for which the default settings are not appropriate should be individually listed in the MANIFEST file.
- The System files can be included in the ZIP file or placed in the S3 bucket, external to the zip. The System files must always be named according to the Zip file that they relate to (eg B9999999_CONTROL.csv & B9999999_MANIFEST.csv)
- The result of Document Import processing can be viewed from the Batch Jobs List (Interfaces -> Find Batch Jobs menu)
- A history note will be written to the project view or order that the documents were attached to specifying that document(s) were added from a document import run and each document added via the bulk import process will identify the specific import batch in the Description column.