Cloud Sync

Cloud Sync is a Pozi Enterprise feature which allows organisations to publish internally-maintained datasets to, with the published version remaining in sync with the local copy.

Geospatial datasets which have been converted to GeoJSON format and are in a WGS84 projection, can be uploaded to an AWS S3 repository (called a bucket), and then a synchronisation process can be established to maintain synchronisation between the original source in a local data folder and the copy in the S3 bucket.

Install AWS Client Tools

For the latest detailed instructions to install AWS Client Tool on Windows please refer to the AWS web site.

Download and install the AWS Client Tools

After installation, check that it's installed successfully.

C:\>aws --version aws-cli/1.16.71 Python/3.6.5 Windows/10 botocore/1.12.61

You should see something similar to  aws-cli/1.16.71 Python/3.6.5 Windows/10 botocore/1.12.61.

Configure AWS Profile

The command below will configure the  default profile. If you are already using AWS client tools on the server you may wish to configure a separate profile. Please refer to the AWS client tools documention for more details about using profiles. The examples below assume the default profile is configured.

C:\>aws configure AWS Access Key ID [None]: <Enter Provided Data> AWS Secret Access Key [None]: <Enter Provided Data> Default region name [ap-southeast-2]: <Press Enter to leave default> Default output format [text]: <Press Enter to leave default>

Configure Windows Task Scheduler

Open Task Scheduler

Windows Key / Click Start -> Type: Task Scheduler -> Press Enter

Create Task

Click  Create Task on the right menu

Basic task data
  • Enter name Pozi Sync
  • Enter description (optional) Sync GeoJSON to AWS S3 for display in Pozi Web GIS
  • Tick radio button Run whether user is logged in or not
Trigger
  • Select Triggers tab
  • Click New button
  • Select Daily
  • Configure other options as required, time of task, etc.
Action
  • Select Actions tab
  • Click New button
  • In Program/script enter the details below replacing the sections with the configuration provided or with your local server setup details such as data folder location.
  • <LOCAL_DATA_FOLDER> is a folder on the server which has the GeoJSON files, e.g. X:\GIS\Export
  • <CUSTOMER_NAME> is the name used by Pozi when loading the web site. E.g. https://gleneira.pozi.com/ the CUSTOMER_NAME would be gleneira
Program/script
aws s3 sync <LOCAL_DATA_FOLDER> s3://<CUSTOMER_NAME>.pozi.com --delete --exclude "*" --include "*.json" --acl public-read

The  --delete flag will delete all files in the bucket that are not found in the <LOCAL_DATA_FOLDER>. That is, if a file is deleted from the folder, it will also be deleted from the bucket when the sync occurs. Without this flag, the files would remain in the bucket even if deleted from the <LOCAL_DATA_FOLDER>.

The  --include flag will only upload .json files. This is to ensure any other temporary files, or files accidentally copied to that folder are not uploaded to our server. This means exported files must be named with a .json extension, if files are exported with .geojson or something else, then adjust this as required. If other files are required, add more --include "*.extension" flags.

The  --acl public-read sets the files as public. That is, anyone who knows the filename will be able to access these files. This is required if authentication is not used.

Testing

Ensure the exported files have been uploaded by loading a web browser and navigating to the web location. For example, if the customer name was gleneira and the filename was somedata.json, then the final file location would be as follows:

https://s3-ap-southeast-2.amazonaws.com/gleneira.pozi.com/somedata.json

Note when going to this link it will likely automatically download the json file for subsequent opening in a text editor, as json files are not normally recognised as a know file type by most browsers.

Please check the contents of the downloaded file to ensure it matches the local version.