The Collection configuration is where you can set up rules to follow while crawling or indexing the data sources. You can conduct a detailed configuration setup with the various sections available here.

You will have access to various technical parameters like indexing, concurrent request, download delay, the definition of allowed domains, specific field settings, and schedule the crawling frequency here. 

Configure Collections Sub Menus

The Keyspider Collection Configuration menu includes the following sub menus:

Configure Collection Sub Menus

Index Configuration

You can add a new index in the section by selecting parameters, assigning relevant values, and save your selection in the ‘New Index’ tab. You can also edit, delete, and enter comments for the selected parameters.

You can search for a specific index through the provided search bar.

The following table describes the Index Configuration parameters.

Adding Index Configuration

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Collection Configuration > Index Configuration. The
   Index Configuration window is displayed.
2. Click Add to add an Index. The Add new configuration window is displayed.
3. Enter or select the parameter values from the drop down. Refer to the table Index Configuration Parameters for parameter description.
4. Click Save to add the Index.

Editing Index Configuration

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Collection Configuration > Index Configuration. The Index Configuration window is displayed.
2. Click on the edit icon to edit the Index Configuration . The Edit Configuration window is
   displayed.
3. Make the updates and click Save to update the configuration.

Deleting Index Configuration

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Collection Configuration > Index Configuration. The Index Configuration window is displayed.
2. Click on the Delete icon to delete the Index Configuration . The Delete configuration window is
   displayed.
3. Click OK to delete the configuration or Cancel to retain the configuration.

Collection Settings

When you wish to conduct changes for a particular collection by mass, the collection settings feature can help you alter/update the existing data sources by parameters here.

Collection Settings Window

The following table describes the Index Configuration parameters.

Adding Collection Settings

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Collection settings. The Collection settings window is displayed.
2. Click Add to add a new collection setting. The Add new configuration window is displayed.
3. Enter or select the parameter values from the drop down. Refer to the table Collection settings Parameters for parameter description.
4. Click Save to add the Collection settings.

Add New Configuration

Editing Configuration

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Collection settings. The Collection settings window is displayed.
2. Click on the edit icon to edit the Configuration . The Edit Configuration window is displayed.
3. Make the updates and click on Save to update the configuration.

Deleting Configuration

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Collection settings. The Collection settings window is displayed.
2. Click on the Delete icon to delete the Configuration . The Delete configuration window is displayed.
3. Click OK to delete the configuration or Cancel to retain the configuration.

Field settings

If you wish to map the fields by file type, you can add a new configuration, set the parameters as field selector, input values, choose the file type, and save the selection.

Field Settings Window

The following table describes the Field Settings parameters.

Adding Field

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Field settings. The Field settings window is displayed.
2. Click Add field to add a new field setting.
3. Enter or select the parameter values from the drop down. Refer to the table Field settings Parameters for parameter description.
4. Click Save to add the Field settings.

Schedule

You can set a definite crawling schedule for the collection by choosing the frequency of the crawling period – daily, weekly, or monthly, specific to your requirements. The default value is set based on the plan you chose.

Schedule Window

Editing Schedule

Following are the steps:

1. In the Keyspider UI, navigate to Collections > Schedule. The Schedule window is displayed.
2. Click on the edit icon to edit the Schedule. The Edit Schedule window is displayed.
3. Make the updates and click on Save to update the configuration.