Import data from Feeds

If you have data stored in web URLs in tabular file formats such as CSV, Excel, JSON, etc., you can easily import them into Analytics Plus. Analytics Plus offers easy import and copy-paste options of such data to jump start your reporting and advanced analysis. 

Note: You can also import data stored in local files, local databases, cloud databases, or Cloud Drives. Click the corresponding links to learn more.

Supported file types

1. What are the various formats in which files can be imported into Analytics Plus?
2. What are the sources from which I can import data into Analytics Plus using the Feeds option?
3. How do I import data from a CSV file into Analytics Plus?
4. How do I import data from Excel into Analytics Plus?
5. How do I import data from Microsoft Access Database files into Analytics Plus?
6. How do I import data from an HTML file into Analytics Plus?
7. How do I import data from JSON into Analytics Plus?
8. How do I import data from XML into Analytics Plus?
9. How do I import data from Statistical Files into Analytics Plus?
10. How do I import data from Parquet files to Analytics Plus?
11. How do I import data using an OData feed?

Supported data locations

1. Is it possible to import data from dynamic URLs?
2. Is it possible to import data into an existing table?
3. Can I import large files into Analytics Plus?
4. Can I import data in batches into Analytics Plus (Pagination)?  

Import Data from Secure Feeds and URLs

1. Is it possible to import data from authenticated URLs into Analytics Plus?
2. How do I import data from URL feeds authenticated via Basic HTTP Authentication?
3. How do I import data from URL feeds authenticated via OAuth 1 Authentication?
4. How do I import data from URL feeds authenticated via OAuth 2 Authentication?
5. How do I import data from URL feeds authenticated via AWS Signature?
6. How do I import data from URL feeds authenticated via Digest Authentication?
7. How do I import data from URL feeds authenticated via Hawk Authentication?

Import Settings

1. How do I set a custom date format?
2. Is it possible to map a column from the source file to a column in an Analytics Plus table while importing?
3. Can I retain the data source's relational data modeling?
4. How do I set On Import Error options?
5. Why do I see Warning Details in the Import Summary?
6. Can I schedule data import?

Supported File Types

1. What are the various formats in which files can be imported into Analytics Plus?

You can import data into Analytics Plus in the following formats:

• Comma Separated Value (CSV)

• Excel 

• MS Access Database Files

• HTML Content

• JSON

• XML

• Statistical Files

• Parquet Files

• OData Feed

If the CSV, TSV, XLS, XLSX, JSON, XML, Spatial, and Parquet files are larger in size, you can compress the files in ZIP or GZIP format and import it into Analytics Plus with ease. Please note that the ZIP or GZIP should contain only one file in any of the above mentioned formats.

2. What are the sources from which I can import data into Analytics Plus using the Files tile?

You can import data from web URLs into Analytics Plus from the Feeds tile in the Import Your Data page.

3. How do I import data from a CSV file into Analytics Plus?

4. How do I import data from Excel into Analytics Plus?

5. How do I import data from Microsoft Access Database files into Analytics Plus?

6. How do I import data from an HTML file into Analytics Plus?

7. How do I import data from JSON into Analytics Plus?

8. How do I import data from XML into Analytics Plus?

9. How do I import data from Statistical Files into Analytics Plus?

10. How do I import data from Parquet Files into Analytics Plus?

11. How do I import data using an OData feed?

Supported data locations

1. Is it possible to import data from dynamic URLs?

You can import data from a dynamic URL using parameters. If the URL from which data is being imported contains current date and time values, this can be filled using the Current Date, Current Time and Current Date & Time parameter value in the first step of the import process. This will import data from URLs that change dynamically based on the date. 

2. Is it possible to import data into an existing table in Analytics Plus?

Data can be imported into existing tables in Analytics Plus in any of the applicable file formats, from the supported data locations. 

Follow the steps below to import data into existing tables. 

• Open the table into which you wish to import additional data.

• Click the Import Data button on the top right corner, and select the Import Data into this Table option. 

• In the Import Your Data page that appears, select the required tile to import data from local files, web URLs or cloud drives. 

• In the page that appears, choose the appropriate option from the How do you to Import? drop down. The following options are available: 

  • Add records at the end: Use this option to append imported records to the end of the table.

  • Delete existing records and add: Use this option to delete all the records in the table and add the new records into the table.

  • Add records and replace if already exists: Use this option to update existing records with the new values and append new records to the end of the table.

  • Add records, replace existing and delete missing record: This option allows you to update the existing records with new values, append new records to the end of the table and delete the records in the table that are not present in the file that is being imported.

Note: When replacing existing records in Analytics Plus, select the columns in the source file that are to be matched to the existing ones in Analytics Plus. The values in these columns will be used as the key to match the existing records in the table to the new records being imported. Thus, the columns should be selected such that they uniquely identify each record in the table. If unique records are not identified, all records will be appended to the end of the table.

• Choose the required file type and data location, select the source file, and click Next to proceed with the import process as usual. 

3. Can I import large files into Analytics Plus? 

Analytics Plus has the following restrictions on data import: 

• You cannot import more than 5 million rows into Analytics Plus using the Import wizard. 

• Files whose sizes exceed 500 MB should be compressed and imported into Analytics Plus.

To can import files larger than 500MB or those that contain more than 5 million rows, use the Analytics Plus Agent. It is a packaged downloadable tool that directly accesses your database, queries the required data and uploads it to Analytics Plus. Scheduled data imports can also be set up. To learn more, refer the Analytics Plus Agent help document.

4. Can I import data in batches into Analytics Plus (Pagination)?  

Yes. You can select the required pagination option from the Parameters section in the Import your data screen. 

Note:

• Pagination options can be used in Analytics Plus only if your API provider provides permission for it. 

• Currently, you can only paginate files in CSV, JSON, and XML formats.

The following Pagination options are available in Analytics Plus: 

• Page Number: Use this option when your data is stored in multiple pages and you wish to import them in the same sequence into Analytics Plus. Provide the following details: 

  • Parameter Name: Enter the page parameter name. You can get this name from your API/URL provider.

  • Initial Value: Provide the initial page number from which you want to start importing. 

  • No. of Requests: Provide the number of requests you wish to execute in sequence. For example, on specifying the number of requests as 10, the request will be executed ten times, and ten pages will be fetched.

• Next Page URL: This option allows you to import data in pages, where each page provides the URL for the next page that is to be imported. Provide the following detail to employ this pagination option: 

  • URL Property Path: Enter the property name that will fetch the URL of the next page. This JSON property will be executed till the property is empty or null. 

Note: The Next Page URL pagination is applicable only for JSON and XML files.

• Offset and Limit: This option allows you to import a large dataset into Analytics Plus in batches, using the batch size of each batch and the starting position.To use the option, provide the following details: 

  • Offset Parameter: Enter the Offset Parameter Name provided by your API provider.

  • Initial Value: Enter the initial offset value.

  • Limit Parameter: Enter the Limit Parameter Name given by your API provider.

  • Limit Value: Provide the total number of records to be fetched from the offset.

  • No. of Requests: Provide the number of sequential requests that need to be carried out.

The specifications provided in the above example will invoke the provided URL as follows: 
http://reports-ubuntu2.csez.zylker.com/Pagination_Data/JSONData?limit=100&offset=0
http://reports-ubuntu2.csez.zylker.com/Pagination_Data/JSONData?limit=100&offset=100 
http://reports-ubuntu2.csez.zylker.com/Pagination_Data/JSONData?limit=100&offset=200 and so forth. For each consequent request, the offset value is calculated by adding the previous request's offset value with the offset limit. 

Import Data from Secure Feeds and URLs

1. Is it possible to import data from authenticated URLs?

Analytics Plus allows you to import data from authenticated URLs that support the following authentication methods:

• HTTP Basic Authentication 
• OAuth 1
• OAuth 2
• AWS Signature
• Digest Authentication
• Hawk Authentication

2. How do I import data from URL feeds authenticated via Basic HTTP Authentication?

3. How do I import data from URL feeds authenticated via OAuth 1 Authentication?

4. How do I import data from URL feeds authenticated via OAuth 2 Authentication?

5. How do I import data from URL feeds authenticated via AWS Signature?

6. How do I import data from URL feeds authenticated via Digest Authentication?

7. How do I import data from URL feeds authenticated via Hawk Authentication?

Import Settings

1. How do I set a custom date format?

By default, Analytics Plus identifies the date format of the date columns in the data being imported. If the default format is incorrect, you can easily modify the same while importing data into Analytics Plus. 

There are two ways to set a custom date format for your date columns.

• Select the required date format from the Format of Date Column (s) drop down. This option will apply the selected date format to all date columns in the data being imported.

• You can also set different date formats for each date column in your data. To do this, click the Settings icon in the preview column header, select the Change Date Format option and set the required date format. 

Refer the Constructing a Custom Date format document to know more. 

2. Is it possible to map a column from the source file to a column in an Analytics Plus table while importing data?

While importing data into an existing table in Analytics Plus, you can map a column from the imported file to a specific column in the Analytics Plus table. In the Data Preview section of the Import wizard, use the drop down in the column header to choose the required column. You can either select the column the data needs to be mapped to, or specify that it is a new column.

3. Can I retain the data source's relational data modeling?

While you cannot retain your source's relational data modeling, Analytics Plus allows you to auto join tables during data import. While importing data into an existing table, Analytics Plus auto identifies similar columns and provides suggestions for lookup. 

The following presentation explains how this works.

4. How do I set On Import Error options?

The On Import Errors drop down in the Import wizard allows you to specify how import errors are to be handled in Analytics Plus. 

The following are the available options:

• Set Empty Value for the Column: This option sets an empty value to columns that had problems while importing.

• Skip Corresponding Rows: This option skips the rows in which an error occurred during data import.

• Don't Import the data: If any errors occur during data import, this option allows you to abort the import process.

In case any error occurs during import, the details of the same would be shown in the Import Summary dialog. Refer the next question to know more. 

5. Why do I see Warning Details in the Import Summary?

Specifying a wrong data type for your column will generate errors on import. Analytics Plus will handle such errors per the specified On Import Errors option and display details of the same in the Import Summary dialog. 

6. Can I schedule data import?

Analytics Plus allows you to schedule the import of data from FTP servers and web URLs. 

Note: To schedule data import, it is mandatory that the data has a column header.