This blog will guide you through indexing content from API endpoints into Coveo using the Generic REST API source. This tool allows the indexing of data from any web service exposing a REST API, such as Content Management Systems (CMS), databases, or third-party platforms. While the demo API here is for illustration, the steps can be adapted for real-world scenarios, like integrating Adobe Experience Manager (AEM) data or fetching structured content, such as user profiles or metadata, from various repositories.
By the end of this guide, you’ll be able to index and display structured data in Coveo, enhancing search relevance and user experience.
Why Use Coveo’s Generic REST API Source?
The Generic REST API source in Coveo is designed to pull data from an API and make it available for search and analytics. It’s highly versatile and enables you to:
- Index data from any API without a dedicated connector.
- Customize indexing with JSON configurations for API calls and responses.
- Access content from various web applications using REST APIs. And handle content from diverse repositories efficiently.
- Simplifies data retrieval and management through structured API calls.
What You Need
- API Access: The API should expose the data you want to index. For this blog, we’re using a sample API.
- Coveo Admin Console Access: You’ll need admin rights to configure sources.
- Optional Tool: Use Postman or a similar API client to inspect API responses before configuration.
Step-by-Step Implementation
Step 1: Explore the API Output
Before integrating with Coveo, examine the API response to understand the data structure. For this blog, I’m using Free fake and reliable API for testing and prototyping.
Test with Postman or cURL:
curl --location --request GET 'https://jsonplaceholder.typicode.com/users'
API Call:
GET https://jsonplaceholder.typicode.com/users Accept: application/json Authorization: Bearer **********-****-****-****-************
Sample API Response:
[
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874"
},
"phone": "1-770-736-8031 x56442",
"website": "hildegard.org"
}
]
JSON Configuration Structure
Before crawling data from the API, define a clear structure for the JSON configuration. This structure determines how Coveo retrieves and organizes data from the API.
- API (Root Node): The interaction entry point, representing the external data source (any RESTful API).
- Service: The layer that processes requests to the API and supports various data sources or services.
- Endpoints: Defined paths the API exposes (e.g., /users, /comments, /albums).
- Data Mapping: Each endpoint retrieves specific data (e.g., user profiles from /users, comments from /comments, albums from /albums).
Identify Fields for Indexing
From the response, fields like id, name, email, and address can be mapped for indexing.
Step 2: Create the Generic REST API Source Connector in coveo
- Log In to Coveo Admin Console: Navigate to Content > Sources and click Add Source.
- Select Source Type: Choose REST API from the options.
- Configure Basic Details, Source Name: Use something descriptive like user profiles etc.
- Don’t build it; just save it for now.
Note:
- REST API URL: https://jsonplaceholder.typicode.com/users
- Authentication: Skip if the API is public. If authentication is required, configure it under the Authentication Tab.
Step 3: Configure JSON Settings
- After saving the source, click Edit JSON Configuration from the source menu.
- Replace the default JSON with the following configuration:
{
"services": [
{
"url": "https://jsonplaceholder.typicode.com",
"paging": {
"pageSize": 20,
"offsetStart": 10,
"offsetType": "item",
"parameters": {
"limit": "limit",
"offset": "offset"
}
},
"endpoints": [
{
"path": "/users",
"method": "GET",
"itemType": "userprofile",
"uri": "%[coveo_url]/users/%[id]",
"clickableUri": "%[coveo_url]/users/%[id]",
"title": "%[name]",
"metadata": {
"id": "%[id]",
"username": "%[username]",
"email": "%[email]",
"address": "%[address]",
"phone": "%[phone]",
"website": "%[website]",
"company": "%[company]"
}
}
]
}
]
}
Step 4: Save and Rebuild the Source
- Click Save to apply changes.
- Go back to the source page and click Save and Rebuild.
- Monitor logs to ensure successful indexing using the log browser.
The screenshot below shows the rebuild process or logs confirming successful indexing
Mapping and Displaying Data
Step 5: Map API Fields to Coveo Fields and then associate it with the rest-api source
- Navigate to Fields in the Admin Console.
- As shown below, create new fields for attributes such as id, name, email, address, and others. The field for ‘name’ has already been created; follow the same process for the other fields.
- Map these fields in the Mappings tab of the source configuration.
For more detailed information on Coveo fields and mappings, refer to the following blogs:
- Part 1: Building the Source and Corresponding Mappings in Coveo Enterprise Search / Blogs / Perficient
- Part 2: Building the Source and Corresponding Mappings in Coveo Enterprise Search / Blogs / Perficient
To view a more detailed field mapping configuration for your source, see the steps below: 
See the below Mapping JSON:
[
{
"content": "%[filename]",
"extractionMethod": "METADATA",
"fieldName": "filename",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-qaut7yluelbrwf5b3x23ea2iji",
"kind": "COMMON",
"rules": [
"%[filename]"
]
},
{
"content": "%[clickableuri]",
"extractionMethod": "METADATA",
"fieldName": "clickableuri",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-qidgh3yqtqgxpt2zarqg5y2fki",
"kind": "COMMON",
"rules": [
"%[clickableuri]"
]
},
{
"content": "%[email]",
"extractionMethod": "METADATA",
"fieldName": "up_email",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-q66t3o5osh6wqdzc4rlwrlsc3m",
"kind": "COMMON",
"rules": [
"%[email]"
]
},
{
"content": "%[phone]",
"extractionMethod": "METADATA",
"fieldName": "up_phone",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-qr7g25pyvh3rl42wtyheppcopy",
"kind": "COMMON",
"rules": [
"%[phone]"
]
},
{
"content": "%[username]",
"extractionMethod": "METADATA",
"fieldName": "up_username",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-sfqu5sxfgk32iarhiaes22chhm",
"kind": "COMMON",
"rules": [
"%[username]"
]
},
{
"content": "%[website]",
"extractionMethod": "METADATA",
"fieldName": "up_website",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-qrti6wae2lzbq4iy7vd5zhkatq",
"kind": "COMMON",
"rules": [
"%[website]"
]
},
{
"content": "%[company]",
"extractionMethod": "METADATA",
"fieldName": "up_company",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-sxfkfclwtarbh3fvrjvtfr2iwe",
"kind": "COMMON",
"rules": [
"%[company]"
]
},
{
"content": "%[address]",
"extractionMethod": "METADATA",
"fieldName": "up_address",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-u6ajgmpuoedwigbmeuzhszkpvy",
"kind": "COMMON",
"rules": [
"%[address]"
]
},
{
"content": "%[id]",
"extractionMethod": "METADATA",
"fieldName": "up_id",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-udpex4lvcj5g7jaynzkwsc2fie",
"kind": "COMMON",
"rules": [
"%[id]"
]
},
{
"content": "%[month]",
"extractionMethod": "METADATA",
"fieldName": "month",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-qrz336tyq4qbvj2qizlqpwkaky",
"kind": "COMMON",
"rules": [
"%[month]"
]
},
{
"content": "%[title:crawler]",
"extractionMethod": "METADATA",
"fieldName": "title",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-r3agbznnaduqkgul7b7hudsarq",
"kind": "COMMON",
"rules": [
"%[title:crawler]"
]
},
{
"content": "%[year]",
"extractionMethod": "METADATA",
"fieldName": "year",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-rbfbf2soa6rzlgai7eygpfkgoq",
"kind": "COMMON",
"rules": [
"%[year]"
]
},
{
"content": "%[mobile]",
"extractionMethod": "METADATA",
"fieldName": "mobilephone",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-rdhx4wtyfy5znb6txno4qvcpba",
"kind": "COMMON",
"rules": [
"%[mobile]"
]
},
{
"content": "%[company]",
"extractionMethod": "METADATA",
"fieldName": "company",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-rdjy3kkqcxbs66phsdmcfdcpou",
"kind": "COMMON",
"rules": [
"%[company]"
]
},
{
"content": "%[permanentid]",
"extractionMethod": "METADATA",
"fieldName": "permanentid",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-rirzsjru43xswhpfwc4c7bkb4m",
"kind": "COMMON",
"rules": [
"%[permanentid]"
]
},
{
"content": "%[coveo_externalurl]",
"extractionMethod": "METADATA",
"fieldName": "relatedlink",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-rjuqiqtvi6fe5kbrwwvbc2sanu",
"kind": "COMMON",
"rules": [
"%[coveo_externalurl]"
]
},
{
"content": "%[office]",
"extractionMethod": "METADATA",
"fieldName": "office",
"id": "ashwinitestorg6zfn7o3t-wktezkeyf53ibe2gryghnjsk6y-xsbc254qtlebiqu4dcqt6rcmn4",
"kind": "COMMON",
"rules": [
"%[office]"
]
}
]
To view the indexed documents with their mapped properties, go to Content > Content Browser in the Coveo Admin Console.
Click the properties option in the top-right corner to view metadata, as shown below.
Here are the properties, as shown below
Step 6: Configure Search Interface
Display the indexed data on the Coveo search page. Use Coveo JSUI Framework components to show the details:
<script id="Default" class="result-template" type="text/html" data-layout="list">
<div class="coveo-result-frame">
<div class="coveo-result-cell" style="vertical-align:top;text-align:center;width:32px;"> <span class="CoveoIcon" data-small="true" data-with-label="false"></span>
<div class="CoveoQuickview"></div>
</div>
<div class="coveo-result-cell" style="vertical-align: top;padding-left: 16px;">
<div class="coveo-result-row" style="margin-top:0;">
<div class="coveo-result-cell" style="vertical-align:top;font-size:16px;" role="heading" aria-level="2">
<a class="CoveoResultLink"></a>
</div>
<div class="coveo-result-cell" style="width:120px;text-align:right;font-size:12px">
<div class="coveo-result-row"> <span class="CoveoFieldValue" data-field="@date" data-helper="date"></span> </div>
</div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoExcerpt"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_id" data-text-caption="Id:" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_email" data-text-caption="Email" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_username" data-text-caption="Username" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_address" data-text-caption="Address" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_phone" data-text-caption="Phone" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_company" data-text-caption="Company" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@up_website" data-text-caption="Website" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@filetype" data-text-caption="File Type" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell"> <span class="CoveoFieldValue" data-field="@source" data-text-caption="Source" style="margin-right:30px;"></span> </div>
</div>
<div class="coveo-result-row" style="margin-top:10px;">
<div class="coveo-result-cell">
<div class="CoveoPrintableUri"></div>
</div>
</div>
<div class="coveo-result-row">
<div class="coveo-result-cell">
<div class="CoveoMissingTerms"></div>
</div>
</div>
</div>
</div>
</script>
Search interface displaying the mapped fields.
Conclusion
Indexing data from APIs like AEM into Coveo using the Generic REST API source is straightforward. Following this guide, you can set up a robust pipeline to fetch, map, and display API data in your Coveo search interface. With the flexibility of JSON configurations and Coveo’s search components, it can deliver powerful, enriched search experiences.
Useful Links
Add a REST API source | Coveo Platform
Use the Log Browser to review indexing logs | Coveo Platform
