Workflow Guide¶

This guide explains how to add, update, and manage content in the AI Knowledge Base.

Prerequisites¶

Conda environment ai-dev set up and activated
All dependencies installed (make setup)
Basic understanding of JSON and schema validation

Quick Start¶

# Activate conda environment
eval "$(/home/justin/miniconda3/bin/conda shell.bash hook)"
conda activate ai-dev

# Validate all data
make validate

# Generate views
make views

# Start development server
make serve

Adding New Content¶

1. Choose the Entity Type¶

Determine which entity type your content belongs to:

providers/ - AI service companies
tools/ - AI applications and services
pricing/ - Pricing information
sustainability/ - Environmental data
prompts/ - AI prompts and templates
glossary/ - Terms and definitions
documents/ - Legal/regulatory documents
tags/ - Classification tags

2. Create the JSON File¶

Create a new JSON file in the appropriate data/<entity>/ directory:

# Example: Adding a new provider
touch data/providers/anthropic.json

3. Follow the Schema¶

Each entity type has a strict JSON schema in schemas/<entity>.json. Required fields include:

Common fields (all entities): - id - Stable identifier (kebab-case) - status - One of: "active", "preview", "deprecated" - version - Semantic version (e.g., "1.0.0") - last_updated - ISO date ("YYYY-MM-DD")

Example provider:

{
  "id": "anthropic",
  "name": "Anthropic",
  "hq_country": "US",
  "website": "https://www.anthropic.com",
  "status": "active",
  "version": "1.0.0",
  "created_at": "2024-08-28",
  "last_updated": "2024-08-28"
}

4. Validate Your Data¶

Always validate before committing:

make validate

If validation fails, fix the errors before proceeding.

5. Generate Views¶

Update the documentation:

make views

This creates markdown tables and detail pages from your JSON data.

Updating Existing Content¶

Method 1: Manual Editing¶

Edit the JSON file directly
Update last_updated to today's date
Bump the patch version (e.g., 1.0.0 → 1.0.1)
Run make validate to check
Run make views to regenerate documentation

Method 2: Using the Update Script¶

For automated updates with changelog tracking:

# Update a field
python scripts/update_item.py --entity tools --id chatgpt --set status=deprecated

# Update multiple fields
python scripts/update_item.py --entity providers --id openai \
  --set website="https://openai.com" \
  --set legal_name="OpenAI, L.L.C." \
  --notes "Updated website and legal name"

# Dry run (see what would change)
python scripts/update_item.py --entity tools --id chatgpt --set status=deprecated --dry-run

The update script automatically: - Bumps the patch version - Updates last_updated timestamp
- Creates a changelog entry - Validates the result

Relationships Between Entities¶

Use IDs to reference related entities:

{
  "id": "chatgpt",
  "provider_id": "openai",
  "pricing_refs": ["chatgpt-pricing"],
  "sustainability_ref": "openai-sustainability",
  "tags": ["conversational-ai", "text-generation"]
}

Schema Validation¶

Each entity type has a JSON Schema that enforces:

Required fields - Must be present
Data types - String, number, boolean, array, object
Formats - URLs, dates, email addresses
Enums - Restricted values (e.g., status values)
Patterns - Regex validation (e.g., version numbers)

Common validation errors:

Missing required fields
Invalid date format (use YYYY-MM-DD)
Invalid URLs (must start with https://)
Wrong status values (use active, preview, or deprecated)
Invalid version format (use semantic versioning like 1.0.0)

Versioning Strategy¶

Use semantic versioning:

Major (1.0.0 → 2.0.0) - Breaking changes, major updates
Minor (1.0.0 → 1.1.0) - New features, backwards compatible
Patch (1.0.0 → 1.0.1) - Bug fixes, small updates

The update script automatically bumps patch versions.

Best Practices¶

Data Quality¶

Always validate before committing changes
Use consistent IDs - kebab-case, stable over time
Verify URLs - ensure all links work
Complete required fields - don't leave required fields empty
Use standard formats - ISO dates, semantic versions

Content Guidelines¶

Be precise - Use specific, accurate descriptions
Stay current - Update information regularly
Link related items - Use ID references for relationships
Tag appropriately - Use consistent, meaningful tags
Document changes - Provide clear changelog notes

File Organization¶

One item per file - Each JSON file contains one entity
Descriptive filenames - Use the entity ID as the filename
Consistent structure - Follow the schema exactly
Sort fields - Keep JSON fields in a consistent order

Testing Your Changes¶

Before submitting changes:

# Validate all data
make validate

# Generate fresh documentation  
make views

# Test the build
make build

# Start local server to preview
make serve

Visit http://127.0.0.1:8000 to preview your changes.

Changelog Management¶

All changes are tracked in data/changelog/. The update script automatically creates changelog entries, but you can also create them manually:

{
  "id": "unique-changelog-id",
  "item_type": "provider",
  "item_id": "openai", 
  "version": "1.0.1",
  "change_type": "update",
  "change_date": "2024-08-28",
  "notes": "Updated website URL and security certifications"
}

Troubleshooting¶

Validation Errors¶

If make validate fails:

Read the error message carefully
Check the schema file for requirements
Verify data types and formats
Ensure all required fields are present

Build Errors¶

If make build fails:

Run make validate first
Check for broken internal links
Verify all referenced files exist
Review MkDocs configuration

Common Issues¶

Missing schema: Ensure schema exists for your entity type
Invalid JSON: Use a JSON validator to check syntax
Broken links: Verify all URLs are accessible
Missing references: Ensure referenced IDs exist

Getting Help¶

Run make help for available commands
Check schema files in schemas/ for requirements
Review existing data files for examples
Use --dry-run flag to preview changes

For deployment instructions, see Cloudflare Pages