Indexing API Authentication

This guide covers implementing authentication for Glean's Indexing API, which powers document indexing, datasource management, and administrative operations. The Indexing API only supports Glean-issued tokens - OAuth authentication is not available.

warning

Critical: The Indexing API does NOT support OAuth authentication. You must use Glean-issued tokens for all Indexing API operations.

Glean-Issued Tokens (Only Option)

Manually created through admin console

Authentication: Glean tokens only (no OAuth support)
Permissions: Global by default (no user-scoping)
Scopes: Full indexing API access (no scope restrictions)
Advanced Features: IP restrictions, token rotation, custom expiry
Use Cases: Document indexing, datasource management, bulk operations

Authentication Headers Reference

The Indexing API uses a simple authentication header format since it only supports Glean-issued tokens:

Authorization: Bearer <indexing_api_token>

Header Details

Header	Required	Description	Example Value
`Authorization`	Always	Bearer token with your Indexing API token	`Bearer glean_XYZ123...`

note

Unlike the Client API, Indexing API tokens do not require additional headers like X-Glean-Auth-Type or X-Glean-ActAs.

Quick Setup Overview

Admin Access Required

Only Super Admins can create Indexing API tokens

Navigate to Token Management

Go to Indexing API Settings

Create New Token

Configure token with optional IP restrictions and rotation settings

Use Token in Requests

Include Authorization: Bearer <token> header in all Indexing API calls

Creating Indexing API Tokens

Prerequisites

Super Admin access to Glean's admin console
Datasource configured in Glean (for document indexing)
IP ranges identified (if using IP restrictions)

Token Creation Process

Navigate to Indexing Tokens

Go to Glean Admin Console → Platform → API Tokens → Indexing Tokens tab

Add New Token

Click "Add API token" and configure:

Token Name: Descriptive name for tracking
Global Permissions: Full indexing API access
App Permissions: Limit to specific datasources (optional)
Expires On: Optional expiration date
IP Restrictions: Optional IP range limitations
Rotation Settings: Optional automated rotation

Save Token Securely

warning

The token is only displayed once after creation. Save it securely - you cannot retrieve it again.

Token Properties

API Support: Indexing API only (Client API not supported)
User Context: Global permissions (no user-scoping available)
Scopes: Full indexing API access (no granular scope control)
Expiration: Optional custom expiry dates
Security: Support for IP restrictions and rotation

Advanced Token Features

IP Address Restrictions

Restrict token usage to specific IP ranges for enhanced security:

IP Restrictions Setup

Configure IP Ranges

During token creation, specify Greenlisted IPs
Use comma-separated list of CIDR format ranges
Example: 192.168.1.0/24,10.0.0.0/16

CIDR Format Examples

CIDR Notation	Description	IP Range
`192.168.1.0/24`	Single subnet	192.168.1.1 - 192.168.1.254
`10.0.0.0/16`	Large network	10.0.0.1 - 10.0.255.254
`203.0.113.5/32`	Single IP	203.0.113.5 only

Benefits

Restrict API access to known infrastructure
Prevent unauthorized usage if token is compromised
Meet compliance requirements for IP-based access control

Token Rotation

Enable automatic token rotation for enhanced security:

Token Rotation Setup

Configure Rotation

During token creation, set Rotation Period (in minutes)
Use the rotation API endpoint to rotate tokens programmatically
Minimum rotation period: 1440 minutes (24 hours)

Using the Rotation API

curl -X POST https://instance-be.glean.com/api/index/v1/rotatetoken \
  -H 'Authorization: Bearer <CURRENT_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "tokenId": "your-token-id"
  }'

Response

{
  "newToken": "new-rotated-token-value",
  "expiresAt": "2024-12-31T23:59:59Z"
}

Best Practices

Implement rotation in your application logic
Store both old and new tokens during rotation period
Test rotation in development environment first

Using Indexing API Tokens

Authentication Header

All Indexing API requests require a single authentication header:

Authorization: Bearer <indexing_api_token>

Example Requests

Index Document
Bulk Index Documents
Get Document Count
Add Datasource

curl -X POST https://instance-be.glean.com/api/index/v1/indexdocument \
  -H 'Authorization: Bearer <INDEXING_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "datasource": "my-datasource",
    "document": {
      "id": "doc-123",
      "title": "Example Document",
      "body": {"mimeType": "text/plain", "textContent": "Document content"},
      "updatedAt": "2024-01-15T10:30:00Z"
    }
  }'

curl -X POST https://instance-be.glean.com/api/index/v1/bulkindexdocuments \
  -H 'Authorization: Bearer <INDEXING_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "datasource": "my-datasource",
    "documents": [
      {
        "id": "doc-1",
        "title": "First Document",
        "body": {"mimeType": "text/plain", "textContent": "Content 1"}
      },
      {
        "id": "doc-2", 
        "title": "Second Document",
        "body": {"mimeType": "text/plain", "textContent": "Content 2"}
      }
    ]
  }'

curl -X GET https://instance-be.glean.com/api/index/v1/getdocumentcount \
  -H 'Authorization: Bearer <INDEXING_TOKEN>' \
  -H 'Content-Type: application/json'

curl -X POST https://instance-be.glean.com/api/index/v1/adddatasource \
  -H 'Authorization: Bearer <INDEXING_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "datasource": {
      "name": "My Custom Datasource",
      "displayName": "Custom Data Source",
      "homeUrl": "https://example.com"
    }
  }'

Testing Your Authentication

Quick Verification

Test your token with a simple document count request:

curl -X GET https://<instance>-be.glean.com/api/index/v1/getdocumentcount \
  -H 'Authorization: Bearer <YOUR_INDEXING_TOKEN>' \
  -H 'Content-Type: application/json'

Expected Response

Successful authentication returns document count data:

{
  "totalDocumentCount": 1234,
  "datasourceDocumentCounts": [
    {
      "datasource": "datasource-1",
      "documentCount": 567
    },
    {
      "datasource": "datasource-2", 
      "documentCount": 667
    }
  ]
}

Test Document Indexing

Test document indexing with a minimal document:

curl -X POST https://<instance>-be.glean.com/api/index/v1/indexdocument \
  -H 'Authorization: Bearer <YOUR_INDEXING_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "datasource": "test-datasource",
    "document": {
      "id": "test-doc-001",
      "title": "Test Document",
      "body": {
        "mimeType": "text/plain",
        "textContent": "This is a test document for authentication verification."
      },
      "updatedAt": "'$(date -u +%Y-%m-%dT%H:%M:%SZ)'"
    }
  }'

Troubleshooting

Common Authentication Errors

Error	Cause	Solution
`401 Unauthorized`	Invalid or expired token	Verify token is correct and not expired
`403 Forbidden - IP not allowed`	Request from non-whitelisted IP	Add your IP to token's allowed IP ranges
`400 Bad Request - Invalid datasource`	Datasource doesn't exist	Create datasource first or verify name
`413 Request Entity Too Large`	Document/request too large	Split into smaller requests or use bulk operations
`429 Too Many Requests`	Rate limit exceeded	Implement exponential backoff retry logic
`500 Internal Server Error`	Glean service issue	Check Glean status page, retry with backoff

Debugging Steps

Verify token validity

Check that token hasn't expired
Ensure token was created for Indexing API (not Client API)
Test with simple endpoint like document count

Check IP restrictions

Verify your IP is in allowed ranges (if IP restrictions enabled)
Test from different IP if needed
Check CIDR format in token configuration

Validate request format

Ensure Content-Type header is set correctly
Verify JSON payload is valid
Check required fields are present

Test with minimal request

Start with document count or datasource status endpoints
Use minimal document for indexing tests
Gradually add complexity once basic auth works

Indexing-Specific Issues

Document Indexing Troubleshooting

Common Document Indexing Issues

Issue	Cause	Solution
Document not appearing in search	Processing delay	Wait 5-10 minutes, check document status API
Permissions error	User lacks access	Verify document permissions are set correctly
Invalid MIME type	Unsupported file format	Check supported MIME types documentation
Document too large	Size exceeds limits	Split document or compress content

Validation Steps

Check document status: Use the document status API to verify indexing completion
Verify permissions: Ensure document permissions allow intended users access
Test search: Search for indexed document using specific terms from content
Monitor logs: Check Glean's admin console for indexing errors

Best Practices

Security

Store tokens securely - never commit tokens to version control
Use IP restrictions when possible to limit token usage
Enable token rotation for long-running applications
Set expiration dates for tokens used in temporary projects
Monitor token usage through the admin console
Create separate tokens for different environments (dev/staging/prod)

Performance

Use bulk operations when indexing multiple documents
Implement retry logic with exponential backoff
Monitor rate limits and adjust request frequency
Batch requests efficiently to reduce API calls
Use appropriate content chunking for large documents

Development

Test thoroughly in development environment first
Use descriptive token names to track purpose and usage
Document token configuration for team members
Plan for token rotation in application architecture
Implement proper error handling for all API responses

Operational

Monitor indexing status regularly through admin console
Set up alerts for indexing failures or errors
Track document counts to verify successful indexing
Regular token audits to remove unused tokens
Backup token configurations for disaster recovery

Next Steps

Setup Datasource

Configure your first datasource for document indexing

Index Documents

Learn document indexing best practices

Indexing API Reference

Explore all Indexing API endpoints

Glean-Issued Tokens (Only Option)

Authentication Headers Reference​

Header Details​

Quick Setup Overview​

Creating Indexing API Tokens​

Prerequisites​

Token Creation Process​

Token Properties​

Advanced Token Features​

IP Address Restrictions​

Token Rotation​

Using Indexing API Tokens​

Authentication Header​

Example Requests​

Testing Your Authentication​

Quick Verification​

Expected Response​

Test Document Indexing​

Troubleshooting​

Common Authentication Errors​

Debugging Steps​

Indexing-Specific Issues​

Best Practices​

Security​

Performance​

Development​

Operational​

Next Steps​