In this section, we outline the protocols CitrusAd supports and how to name files so CitrusAd can download the data files automatically from your server.

Protocols

CitrusAd supports several ways to retrieve data files. Data files should be stored on your server, and files will be downloaded via one of the standard protocols.

Currently, CitrusAd supports the download of data files via the following protocols:

GCS
SFTP
SCP
HTTPS

In general, you will need to provide information about the protocol, host, port, and file path of a data file so CitrusAd can download it. When authentication is required to download data files, you will need to provide CitrusAd with a credential (e.g. username and password) to authenticate with your system.

If you are using an SFTP protocol, CitrusAd supports two types of authentication to download data files:

A username and a password
CitrusAd's public key.

In the second type of authentication, the below public key needs to be installed into your SFTP server.

Staging:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDMd4w5Z6hsjlUc0/r3dTJ+IaM9Mm+l3ZTrX3D5+30ZNiAb3Mhvlp8gv37CFKzqM90oFejl15bbr5dymjycsOWInxqkMyLOTbdMFAk7hmicNvN5bt3tx/Z0UewNvAx0Uw9Qxu7SCMSAalk9l7dqLp5RFFj+RHq9ApV8KIA/Fw2mjCw2vrJhiRWeuUuNNQTbGFBA5y72ATeYA2p0ZK3/TTJxyxCL70zGVxFG0jdAurfXKujYaF73Ql/gl5wmtnxGvTORtvObQdc2jCTsynj1w17LSZAdeoPQGnQkVCk37ZUnWTw8s4RYjHFVunsTDxJO5Qnw5yj8oqhe5IdpQyavUj+otRd6900q/3dK60IELH6AFz+ZEqudlBh8ZO2hvHIDS2Ii5S3LunlejykAVdRl7RVHA17Pk3mmrHCS6i3n9ZliBYeVQiKme2jHVB+ESrHFxQWl3ct9MYFNlAqwuhTFQRSG9ioSEHX5Dnmqk8aqbm/4pYY4b7YHOcIpHrtrVrFTJ3vLM1If1OVrDQidFhi1ArmoCU+PO65jNSeMj27m/HoX7Mbcmi8F++Op3MgP8iJrohE0Wjz+use7nlDtQTtDxDeEQQYGqsz4DOtH/jn1TWRynaXZqwPqC5Q6b52QM22T+E3rgT/4G7yJU3IRKioByOck251MEtOEgSCvvIAaV685fQ==

Production

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCtpIUFoYu8OUesZGP9rffv6/hky1HbQCAhC8hNRmcWpdptzRYsKIZFPz9t2vb85dierTZqJstAEJkePHd25cZ8btIBacbAR49avCIXiBuVgvlJk1+Er2dyYqSzSeFhpzlZxi4CkGo/Nsx7yxPi5693fdVF3vr+/iencw1tdLJJXfT6j1YLBCREg+IziBCO6m9P0QQGEDXYvsOmIzl0CJ8enU7gjvnYL5ify1/aGpoXv9/XcCYJWXTQmvmISknOMdC4pBiuDqsE9CX27FLQQb1ZT5jNrqD5hFsEo0rWqLiCbb5qoxIdMSe/IbA+sewgpgTPNJXeCNLSSH/MHhj2IgiMZGDDDaVaqMZD7N2jeQdI6SQbgvXiLTTUL4gNyO5eN9oqoeJumeZMi0+MQ7kkfHDYGtKe3RMxL7heL650HL+eWDIybezyPdQmfyVew9KuN0uB+2X+2NxQs8ZLqow486XDfUDfo+rPuoIvkYy7Z8ox+gne+1iAk43TCekcDwxlc56R9muMtGLuzzg1x9s9MqsAg5JNWEPQ8giAwCQJwm7h3UcCY3TJCn0LUzYHc7Ycr11JfIXL47vYGH3C0XBYlbEU/8m4VhhexzgJi8G/reclY5pfjtwQGOiobsxLsWgGx6Qoz/qWqmGwQIg+qi7jvA7jnltUP+RTFZpmd18/dSxB+w==

Production:

Data compression and encryption

You may need to compress and encrypt data before uploading it to your server for syncing. When both compression and encryption are used on data files, CitrusAd assumes that the data files are compressed prior to encryption.

When a data file is encrypted, CitrusAd will decrypt the data file after it’s been downloaded. Currently, we support data decryption for files that are encrypted by PGP programs. More information about PGP cryptography can be found at https://tools.ietf.org/html/rfc4880.

If you choose to use this type of cryptography for data files, you can use the below public key sto encrypt data files before uploading the files to your server. Only CitrusAd can decrypt the encrypted data files.

Staging:

Production:

When a data file is delivered compressed, we will decompress the data file prior to processing. Currently, we support decompression of two types of compression formats: zip and gzip.

File naming convention

As mentioned previously, you will need to provide information about the protocol, host, port, and file path to a data file so CitrusAd can download and process the data file. You can choose a daily time that’s convenient to ensure that the data file is ready on the server each day, and CitrusAd will download the data files daily.

The simplest way to specify a target file for CitrusAd to download each day is through providing an explicit file name. CitrusAd will then use this file name to retrieve the data file from your server.

When you use FTP, FTPS, and SFTP protocols for communication between your server and CitrusAd, we support other options for you to specify target files for CitrusAd to download, including:

Rolling_earliest
Rolling_earliest_24_hours
Rolling_latest
Rolling_latest_24_hours.

They are also called target file modes.

When you choose one of the options outlined above, you will need to provide CitrusAd with a textual template for the data file names. In the textual template, there is a special string, which is “{*}”. CitrusAd will use the template provided by you to match the file names on your server to choose and download a target file each day.

An example of a template can be “CitrusAdCatalogDataAU{}.txt”. The template defines that the matched file names must start with the prefix “CitrusAdCatalogData_AU” and end with the suffix “.txt”. When the template “CitrusAdCatalogDataAU{_}.txt” is used, the below file names will match with the template:

CitrusAdCatalogProduct_AU_20190315.txt
CitrusAdCatalogProduct_AU_20190314.txt
CitrusAdCatalogProduct_AU_20190312.txt

In order to avoid downloading data files that you are uploading, CitrusAd only downloads data files that have been modified more than one minute from the time we access the server.

Although there are several file names that match the template, CitrusAd only downloads and processes one file at a time. In order to choose a file from a list of candidates, we define different target file modes you can choose from. Target file modes are discussed in more detail below.

Rolling_earliest

In this target file mode, we use the template to filter files by using their names, then we sort the results by ascending file names and return the first result.

For example, if the template for file names is “CitrusAdCatalogDataAU{*}.txt” and the list of file names filtered by the template are as below, the file “CitrusAdCatalogProduct_AU_20190312.txt” will be chosen to download in this target file mode.

CitrusAdCatalogProduct_AU_20190312.txt
CitrusAdCatalogProduct_AU_20190313.txt
CitrusAdCatalogProduct_AU_20190314.txt

Rolling_earliest_24_hours

In this target file mode, we first use the template to filter files using their names. Then we only choose files that are modified within the recent twenty-four hours. Finally, we sort the results by ascending file names and return the first result.

For example, we assume that the current time is 2019-03-15 10:30:07 and the template for file names is “CitrusAdCatalogDataAU{*}.txt”. If the list of file names that are filtered by the template is in the table below, the file “CitrusAdCatalogProduct_AU_20190314.txt” will be chosen to download in this target file mode.

An example of files with names and last modified information

File name	Last modified
CitrusAdCatalogProduct_AU_20190312.txt	13/03/19 15:35:11
CitrusAdCatalogProduct_AU_20190313.txt	13/03/19 15:35:08
CitrusAdCatalogProduct_AU_20190314.txt	14/03/19 15:35:10
CitrusAdCatalogProduct_AU_20190315.txt	15/03/19 10:05:07

Rolling_latest

In this target file mode, we use the template to filter files using their names. Then we sort the results by descending file names and return the first result.

For example, if the template for file names is “CitrusAdCatalogDataAU{*}.txt” and the list of file names filtered by the template is as below, the file “CitrusAdCatalogProduct_AU_20190314.txt” will be chosen to download in this target file mode.

CitrusAdCatalogProduct_AU_20190314.txt
CitrusAdCatalogProduct_AU_20190313.txt
CitrusAdCatalogProduct_AU_20190312.txt

This target file mode is similar to Rolling_earliest. However, instead of sorting the files by ascending file names, we sort them by descending file names.

Rolling_latest_24_hours

In this target file mode, we first use the template to filter files by using their names. Then we only choose files which are modified within the recent twenty four hours. Finally, we sort the results by descending file names and return the first result.

For example, we assume that the current time is 2019-03-15 10:30:07 and the template for file names is “CitrusAdCatalogDataAU{*}.txt”. If the list of file names that are filtered by the template is in the table below, the file “CitrusAdCatalogProduct_AU_20190315.txt” will be chosen to download in this target file mode.

This target file mode is similar to Rolling_earliest_24_hours. However, instead of sorting the files by ascending file names, we sort them by descending file names.

An example of files with names and last modified information

File name	Last modified
CitrusAdCatalogProduct_AU_20190312.txt	13/03/19 15:35:11
CitrusAdCatalogProduct_AU_20190313.txt	13/03/19 15:35:08
CitrusAdCatalogProduct_AU_20190314.txt	14/03/19 15:35:10
CitrusAdCatalogProduct_AU_20190315.txt	15/03/19 10:05:07