For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

You can re-index or set up auto-indexing to schedule your data sources as needed.

Re-Index Data Sources Using the Control Panel

Re-indexing data sources embeds any new data, which consumes tokens based on the knowledge base’s embeddings model. Buckets, folders, and URLs are updated. Uploaded files aren’t updated. To update uploaded files, remove the files, and re-add it.

You aren’t charged for removed data or indexing events that create no new embeddings. Exact indexing costs are available only after the job completes. For estimates, see knowledge base pricing.

Re-indexing may not fix skipped or failed files unless you resolve the underlying issues first. Review indexing details before retrying.

To reindex data sources, click the Activity tab, under the Activity section, and then on the right of the most recent indexing job, click Re-run to open the Confirm sources update window.

Click Update sources to reindex the data. You are only charged for any new data found during the indexing. You can view the results of the reindexing job in the Activity tab.

Re-Index Data Sources via the API

To reindex data sources via the API, create an indexing job with the knowledge base ID and data source ID. Use the Create Indexing Job endpoint to start the indexing job.

curl -X POST \
  -H "Content-Type: application/json"  \
  -H "Authorization: Bearer $DIGITALOCEAN_TOKEN" \
  "https://api.digitalocean.com/v2/gen-ai/indexing_jobs" \
  -d '{
    "knowledge_base_uuid": "9758a232-b351-11ef-bf8f-4e013e2ddde4",
    "data_source_uuids": [
      "9a825ee0-bbb1-11ef-bf8f-4e013e2ddde4"
    ]
  }'

To check the indexing job’s progress, use the Get Indexing Job endpoint.

After indexing completes, use the Get Knowledge Base endpoint to confirm completion and review the final token count and indexing cost.

If the job takes longer than expected, cancel it using the Cancel Indexing Job endpoint, and then restart it. If issues persist, contact support for assistance.

Schedule Indexing Data Sources Using the Control Panel

You can schedule auto-indexing to keep your knowledge base up to date. Indexing jobs may incur costs when they process new data and create embeddings. If no changes are detected, the job completes with no changes and you are not billed.

To set up auto-indexing, on the right, click Schedule Indexing to open the Create Indexing Schedule window.

Under the Days section, select the days you want indexing to run.

Under the Trigger Time section, set the time of day using the Hrs and Mins dropdown lists. Scheduling time is in UTC.

Then, click Create Indexing Schedule.

Your schedule appears under the Auto-Indexing sub-section, showing its current schedule, the next scheduled run, the last indexing job (manual or scheduled), whether the last job is in progress, complete, or failed, and when the schedule was created.

Scheduled jobs are skipped if they overlap with manual or re-indexing jobs. Failed scheduled jobs don’t cancel the schedule.

You can view your indexing job logs in the Activity tab.

Manage a Indexing Schedule Using the Control Panel

To manage your indexing schedule, on the right of your schedule, click …, and then you can either click:

• Pause Indexing to pause auto-indexing.
• Resume Indexing to resume auto-indexing.
• Edit Schedule to open the Update Indexing Schedule window where you can update the days and the time your auto-indexing runs.
• Destroy to open the Remove Scheduled Indexing window. In the textbox, enter “delete” to confirm deletion, and then click Destroy.

Schedule Indexing Data Sources via the API

To schedule indexing via the API, create an indexing schedule with the knowledge base ID, the time of day you want the job to run (in UTC, using 24-hour format), and the days of the week to schedule the runs, where days are numbered 1 (Monday) through 7 (Sunday). Use the Create Scheduled Indexing Job endpoint.

curl --location --request POST 'https://api.digitalocean.com/v2/gen-ai/scheduled-indexing' \
--header 'Authorization: Bearer $DIGITALOCEAN_TOKEN"' \
--header 'Content-Type: application/json' \
--data '{
    "knowledge_base_uuid": "knowledge_base_uuid",
    "time": "18:00",
    "days": [
        1,
        2,
        3
    ]
}'

After you set up auto-indexing, use the List Scheduled Indexing Jobs for a Knowledge Base endpoint to verify the schedule on your knowledge base.

If you no longer need the schedule, use the Delete Scheduled Indexing Job endpoint. Deleting a schedule doesn’t affect existing knowledge base data or previously completed indexing jobs.