markitect-main/docs/graphql_interface.md

# GraphQL Interface for MarkiTect

## Overview

The GraphQL interface provides a powerful, type-safe way to query and modify MarkiTect's database content including Markdown files, ASTs, schemas, and metadata. This interface enables developers to build rich applications on top of MarkiTect's data with flexible querying capabilities and complete CRUD operations.

## Features

- **Complete Type Safety**: Strongly-typed GraphQL schema with comprehensive field definitions
- **Flexible Querying**: Query exactly the data you need with GraphQL's selective field syntax
- **Rich Data Access**: Access markdown files, schemas, ASTs, and computed fields
- **Search Capabilities**: Full-text search across files and schemas with relevance scoring
- **Pagination Support**: Efficient pagination for large datasets
- **Real-time Introspection**: Schema introspection and GraphQL Playground for development
- **Multiple Access Methods**: HTTP server, local execution, and CLI integration
- **Full CRUD Operations**: Create, read, update, and delete markdown files and schemas via mutations
- **Structured Error Handling**: Comprehensive error responses with validation and success indicators

## Getting Started

### Prerequisites

Install the required dependencies:

```bash
pip install graphene flask flask-cors jsonpath-ng requests
```

### Starting the GraphQL Server

Start the GraphQL server using the CLI:

```bash
markitect graphql-serve
```

By default, this starts the server on `http://localhost:5000`. You can customize the host and port:

```bash
markitect graphql-serve --host 0.0.0.0 --port 8080
```

### Accessing the GraphQL Playground

Once the server is running, visit `http://localhost:5000/graphql` in your browser to access the GraphQL Playground, an interactive development environment for testing queries.

## GraphQL Schema

### Core Types

#### MarkdownFile
Represents a Markdown file in the MarkiTect database:

```graphql
type MarkdownFile {
  id: Int!
  filename: String!
  content: String
  frontMatter: [FrontMatter]
  frontMatterRaw: JSONString
  createdAt: DateTime
  wordCount: Int
  lineCount: Int
  hasFrontMatter: Boolean
}
```

#### Schema
Represents a JSON schema:

```graphql
type Schema {
  id: Int!
  filename: String!
  title: String
  description: String
  schemaContent: JSONString!
  createdAt: DateTime
  updatedAt: DateTime
  schemaVersion: String
  propertyCount: Int
}
```

#### AST
Represents an Abstract Syntax Tree for a Markdown file:

```graphql
type AST {
  fileId: Int
  filename: String!
  tree: [ASTNode]
  metadata: JSONString
  headingCount: Int
  linkCount: Int
  imageCount: Int
  codeBlockCount: Int
}
```

#### DatabaseStats
Provides statistics about the MarkiTect database:

```graphql
type DatabaseStats {
  totalFiles: Int
  totalSchemas: Int
  totalSizeBytes: Int
  lastUpdated: DateTime
}
```

#### SearchResult
Represents search results across files and schemas:

```graphql
type SearchResult {
  type: String!
  score: Float
  file: MarkdownFile
  schema: Schema
  highlight: String
}
```

### Query Operations

#### Single Item Queries

Get a specific markdown file:
```graphql
{
  markdownFile(id: 1) {
    id
    filename
    content
    wordCount
    hasFrontMatter
  }
}
```

Get a specific schema:
```graphql
{
  schema(filename: "user-schema.json") {
    title
    description
    schemaVersion
    propertyCount
  }
}
```

Get AST for a file:
```graphql
{
  ast(filename: "document.md") {
    headingCount
    linkCount
    tree {
      type
      value
      level
    }
  }
}
```

#### List Queries

Get all markdown files with pagination:
```graphql
{
  markdownFiles(limit: 10, offset: 0) {
    id
    filename
    createdAt
    hasFrontMatter
  }
}
```

Filter files by front matter presence:
```graphql
{
  markdownFiles(hasFrontMatter: true, limit: 5) {
    filename
    frontMatter {
      key
      value
    }
  }
}
```

Get all schemas:
```graphql
{
  schemas(limit: 20) {
    filename
    title
    description
    propertyCount
  }
}
```

#### Search Queries

Search across all content:
```graphql
{
  search(query: "documentation", type: "all", limit: 10) {
    type
    score
    file {
      filename
      content
    }
    schema {
      title
      description
    }
    highlight
  }
}
```

Search only in files:
```graphql
{
  search(query: "API", type: "file", limit: 5) {
    score
    file {
      filename
      wordCount
    }
    highlight
  }
}
```

#### Database Statistics

Get database overview:
```graphql
{
  databaseStats {
    totalFiles
    totalSchemas
    totalSizeBytes
    lastUpdated
  }
}
```

#### Advanced AST Queries

Query AST using JSONPath expressions:
```graphql
{
  astQuery(
    filename: "document.md",
    jsonpath: "$.children[?(@.type=='heading')]"
  )
}
```

## Mutation Operations

The GraphQL interface supports full CRUD operations for markdown files and schemas through mutations. All mutations return structured payloads with success indicators and error handling.

### Mutation Types

#### Markdown File Mutations

##### Add Markdown File
Create a new markdown file in the database:

```graphql
mutation {
  addMarkdownFile(
    filename: "new-document.md"
    content: "# New Document\n\nThis is a new markdown file."
  ) {
    success
    markdownFile {
      id
      filename
      content
      wordCount
      hasFrontMatter
      createdAt
    }
    errors
  }
}
```

##### Add Markdown File with Front Matter
Create a file with YAML front matter:

```graphql
mutation {
  addMarkdownFile(
    filename: "blog-post.md"
    content: """---
title: "My Blog Post"
date: "2024-01-15"
tags: ["tech", "api"]
---

# My Blog Post

Content goes here.
"""
  ) {
    success
    markdownFile {
      id
      filename
      frontMatter {
        key
        value
      }
      hasFrontMatter
    }
    errors
  }
}
```

##### Update Markdown File
Update an existing markdown file:

```graphql
mutation {
  updateMarkdownFile(
    id: 1
    content: "# Updated Document\n\nThis content has been updated."
  ) {
    success
    markdownFile {
      id
      filename
      content
      wordCount
      lineCount
    }
    errors
  }
}
```

#### Schema Mutations

##### Add JSON Schema
Create a new JSON schema:

```graphql
mutation {
  addSchema(
    filename: "user-schema.json"
    schemaContent: {
      "type": "object",
      "title": "User Schema",
      "description": "Schema for user objects",
      "properties": {
        "name": {"type": "string"},
        "email": {"type": "string", "format": "email"},
        "age": {"type": "integer", "minimum": 0}
      },
      "required": ["name", "email"]
    }
  ) {
    success
    schema {
      id
      filename
      title
      description
      propertyCount
      schemaVersion
    }
    errors
  }
}
```

##### Update JSON Schema
Update an existing schema:

```graphql
mutation {
  updateSchema(
    id: 1
    schemaContent: {
      "type": "object",
      "title": "Updated User Schema",
      "description": "Updated schema for user objects",
      "properties": {
        "name": {"type": "string"},
        "email": {"type": "string", "format": "email"},
        "age": {"type": "integer", "minimum": 0},
        "country": {"type": "string"}
      },
      "required": ["name", "email"]
    }
  ) {
    success
    schema {
      id
      filename
      title
      propertyCount
      updatedAt
    }
    errors
  }
}
```

##### Delete Schema
Remove a schema from the database:

```graphql
mutation {
  deleteSchema(filename: "old-schema.json") {
    success
    deletedFilename
    errors
  }
}
```

### Mutation Payload Types

All mutations return structured payload types with the following pattern:

#### AddMarkdownFilePayload
```graphql
type AddMarkdownFilePayload {
  success: Boolean!
  markdownFile: MarkdownFile
  errors: [String!]!
}
```

#### UpdateMarkdownFilePayload
```graphql
type UpdateMarkdownFilePayload {
  success: Boolean!
  markdownFile: MarkdownFile
  errors: [String!]!
}
```

#### AddSchemaPayload
```graphql
type AddSchemaPayload {
  success: Boolean!
  schema: Schema
  errors: [String!]!
}
```

#### UpdateSchemaPayload
```graphql
type UpdateSchemaPayload {
  success: Boolean!
  schema: Schema
  errors: [String!]!
}
```

#### DeleteSchemaPayload
```graphql
type DeleteSchemaPayload {
  success: Boolean!
  deletedFilename: String
  errors: [String!]!
}
```

### Error Handling in Mutations

Mutations provide comprehensive error handling:

#### Validation Errors
Invalid input data returns structured errors:

```graphql
mutation {
  addMarkdownFile(filename: "", content: "test") {
    success  # false
    markdownFile  # null
    errors  # ["Filename cannot be empty"]
  }
}
```

#### Duplicate Filename Handling
Attempting to create files with existing names:

```graphql
mutation {
  addMarkdownFile(filename: "existing.md", content: "test") {
    success  # false
    errors  # ["File with filename 'existing.md' already exists"]
  }
}
```

#### Not Found Errors
Updating non-existent resources:

```graphql
mutation {
  updateMarkdownFile(id: 999, content: "test") {
    success  # false
    errors  # ["Markdown file with ID 999 not found"]
  }
}
```

### Complete CRUD Workflow Example

Here's a complete example showing the full lifecycle of a markdown file:

```graphql
# 1. Create a new file
mutation CreateFile {
  addMarkdownFile(
    filename: "api-guide.md"
    content: """---
title: "API Usage Guide"
version: "1.0"
---

# API Usage Guide

Basic API documentation.
"""
  ) {
    success
    markdownFile {
      id
      filename
      hasFrontMatter
    }
    errors
  }
}

# 2. Query the created file
{
  markdownFile(filename: "api-guide.md") {
    id
    content
    wordCount
    frontMatter {
      key
      value
    }
  }
}

# 3. Update the file content
mutation UpdateFile {
  updateMarkdownFile(
    id: 1
    content: """---
title: "API Usage Guide"
version: "2.0"
updated: "2024-01-15"
---

# API Usage Guide

Comprehensive API documentation with examples.

## Authentication
...
"""
  ) {
    success
    markdownFile {
      id
      wordCount
      lineCount
      frontMatter {
        key
        value
      }
    }
    errors
  }
}

# 4. Verify the update
{
  markdownFile(id: 1) {
    filename
    wordCount
    frontMatter {
      key
      value
    }
  }
}
```

## CLI Integration

### Available Commands

#### graphql-serve
Start the GraphQL server:
```bash
markitect graphql-serve [OPTIONS]

Options:
  --host TEXT     Host to bind to (default: 127.0.0.1)
  --port INTEGER  Port to bind to (default: 5000)
  --debug         Enable debug mode
  --no-cors       Disable CORS
```

#### graphql-query
Execute GraphQL queries from the command line:
```bash
markitect graphql-query QUERY [OPTIONS]

Options:
  --variables TEXT    JSON string of query variables
  --endpoint TEXT     GraphQL endpoint URL
  --local            Execute query locally without HTTP
  --format [json|yaml|table]  Output format (default: json)
```

Examples:
```bash
# Execute a simple query locally
markitect graphql-query "{ databaseStats { totalFiles } }" --local

# Execute query with variables
markitect graphql-query "query($limit: Int) { markdownFiles(limit: $limit) { filename } }" --variables '{"limit": 5}' --local

# Execute query against remote server
markitect graphql-query "{ schemas { title } }" --endpoint http://localhost:5000/graphql
```

#### graphql-mutate
Execute GraphQL mutations from the command line:
```bash
markitect graphql-mutate MUTATION [OPTIONS]

Options:
  --variables TEXT    JSON string of mutation variables
  --endpoint TEXT     GraphQL endpoint URL
  --local            Execute mutation locally without HTTP
  --format [json|yaml|table]  Output format (default: json)
```

Examples:
```bash
# Add a new markdown file locally
markitect graphql-mutate 'mutation { addMarkdownFile(filename: "test.md", content: "# Test") { success markdownFile { id filename } errors } }' --local

# Add markdown file with variables
markitect graphql-mutate 'mutation($filename: String!, $content: String!) { addMarkdownFile(filename: $filename, content: $content) { success errors } }' --variables '{"filename": "doc.md", "content": "# Documentation"}' --local

# Update a file remotely
markitect graphql-mutate 'mutation { updateMarkdownFile(id: 1, content: "# Updated") { success errors } }' --endpoint http://localhost:5000/graphql

# Add a JSON schema
markitect graphql-mutate 'mutation { addSchema(filename: "user.json", schemaContent: {"type": "object", "properties": {"name": {"type": "string"}}}) { success schema { id title } errors } }' --local

# Delete a schema
markitect graphql-mutate 'mutation { deleteSchema(filename: "old-schema.json") { success deletedFilename errors } }' --local
```

#### graphql-schema
Get the GraphQL schema definition:
```bash
markitect graphql-schema [OPTIONS]

Options:
  --format [sdl|json]  Schema format (default: sdl)
  --endpoint TEXT      GraphQL endpoint URL
```

#### graphql-examples
Show example queries and usage:
```bash
markitect graphql-examples
```

## Programming API

### Using the GraphQL Client

```python
from markitect.graphql import GraphQLClient

# Create a client
client = GraphQLClient("http://localhost:5000/graphql")

# Execute a query
result = client.execute("""
{
  markdownFiles(limit: 5) {
    filename
    wordCount
  }
}
""")

print(result['data'])
```

### Local Execution

```python
from markitect.graphql import GraphQLClient

# Execute queries locally without HTTP
client = GraphQLClient()
result = client.execute_local("""
{
  databaseStats {
    totalFiles
    totalSchemas
  }
}
""")

print(result['data'])
```

### Executing Mutations

```python
from markitect.graphql import GraphQLClient

# Create a client
client = GraphQLClient("http://localhost:5000/graphql")

# Add a new markdown file
create_mutation = """
mutation {
  addMarkdownFile(
    filename: "api-docs.md"
    content: "# API Documentation\\n\\nComprehensive API guide."
  ) {
    success
    markdownFile {
      id
      filename
      wordCount
    }
    errors
  }
}
"""

result = client.execute(create_mutation)
if result['data']['addMarkdownFile']['success']:
    file_id = result['data']['addMarkdownFile']['markdownFile']['id']
    print(f"Created file with ID: {file_id}")
else:
    print("Errors:", result['data']['addMarkdownFile']['errors'])

# Update the file
update_mutation = """
mutation UpdateFile($id: Int!, $content: String!) {
  updateMarkdownFile(id: $id, content: $content) {
    success
    markdownFile {
      wordCount
      lineCount
    }
    errors
  }
}
"""

variables = {
    "id": file_id,
    "content": "# API Documentation\\n\\nUpdated comprehensive API guide with examples."
}

result = client.execute(update_mutation, variables=variables)
print("Update result:", result['data']['updateMarkdownFile']['success'])
```

### Local Mutation Execution

```python
from markitect.graphql import GraphQLClient

# Execute mutations locally without HTTP
client = GraphQLClient()

# Add a JSON schema locally
schema_mutation = """
mutation {
  addSchema(
    filename: "product-schema.json"
    schemaContent: {
      "type": "object",
      "title": "Product Schema",
      "properties": {
        "name": {"type": "string"},
        "price": {"type": "number", "minimum": 0},
        "category": {"type": "string"}
      },
      "required": ["name", "price"]
    }
  ) {
    success
    schema {
      id
      title
      propertyCount
    }
    errors
  }
}
"""

result = client.execute_local(schema_mutation)
if result['data']['addSchema']['success']:
    schema_info = result['data']['addSchema']['schema']
    print(f"Created schema: {schema_info['title']} with {schema_info['propertyCount']} properties")
```

### Using the Server Directly

```python
from markitect.graphql import GraphQLServer

# Create and start server
server = GraphQLServer(db_path="/path/to/markitect.db")
app = server.create_app()

# Start server
server.run(host="0.0.0.0", port=8080, debug=True)
```

## API Endpoints

When running the GraphQL server, the following endpoints are available:

- `POST /graphql` - Main GraphQL endpoint for queries
- `GET /graphql` - GraphQL Playground interface
- `GET /schema` - Schema introspection
- `GET /health` - Health check endpoint

### Health Check Response

```json
{
  "status": "healthy",
  "database": "connected",
  "database_path": "/path/to/markitect.db"
}
```

## Error Handling

The GraphQL interface provides comprehensive error handling:

### Query Errors
Invalid GraphQL syntax or non-existent fields return structured errors:

```json
{
  "data": null,
  "errors": [
    {
      "message": "Cannot query field 'nonexistentField' on type 'Query'"
    }
  ]
}
```

### Database Errors
Database connectivity issues are handled gracefully:

```json
{
  "data": {
    "databaseStats": null
  },
  "errors": [
    {
      "message": "Database connection error"
    }
  ]
}
```

## Performance Considerations

### Pagination
Always use pagination for large datasets:

```graphql
{
  markdownFiles(limit: 50, offset: 0) {
    id
    filename
  }
}
```

### Field Selection
Only query the fields you need:

```graphql
# Good - selective fields
{
  markdownFiles {
    id
    filename
  }
}

# Avoid - requesting large content fields unnecessarily
{
  markdownFiles {
    id
    filename
    content  # Only include if needed
  }
}
```

### Search Optimization
Use specific search types when possible:

```graphql
# Better - search only files
{
  search(query: "API", type: "file", limit: 10) {
    file {
      filename
    }
  }
}

# Less efficient - search all types
{
  search(query: "API", type: "all", limit: 10) {
    type
    file { filename }
    schema { title }
  }
}
```

## Security Considerations

### Write Operations Security
The GraphQL interface supports both read and write operations through mutations. Write operations require careful consideration:

- **Data Validation**: All mutations include comprehensive input validation and structured error responses
- **Transaction Safety**: Database operations are wrapped in transactions to ensure data consistency
- **File Overwrites**: Duplicate filenames in the same mutation will overwrite existing files
- **Schema Validation**: JSON schemas are validated before storage to ensure they are well-formed
- **Error Handling**: Failed operations return detailed error messages without exposing internal system details

### CORS Configuration
CORS is enabled by default for browser access. Disable with `--no-cors` if needed:

```bash
markitect graphql-serve --no-cors
```

### Database Access
The interface uses the same database path configuration as the main MarkiTect CLI, respecting the `MARKITECT_DB` environment variable.

## Troubleshooting

### Common Issues

#### "Flask is required" Error
Install Flask dependencies:
```bash
pip install flask flask-cors
```

#### "requests is required" Error
Install requests for HTTP client functionality:
```bash
pip install requests
```

#### Database Connection Errors
Check that your MarkiTect database exists and is accessible:
```bash
# Check database location
echo $MARKITECT_DB

# Verify database exists
ls -la ~/.markitect/markitect.db
```

#### GraphQL Syntax Errors
Use the GraphQL Playground at `http://localhost:5000/graphql` to validate your queries with real-time syntax checking and auto-completion.

## Examples

### Complete Data Export
Export all markdown files with metadata:

```graphql
{
  markdownFiles {
    id
    filename
    content
    createdAt
    wordCount
    lineCount
    hasFrontMatter
    frontMatter {
      key
      value
    }
  }
}
```

### Schema Analysis
Analyze all schemas in the database:

```graphql
{
  schemas {
    filename
    title
    description
    schemaVersion
    propertyCount
    schemaContent
  }
}
```

### Content Discovery
Find all documents containing specific terms:

```graphql
{
  search(query: "authentication security", type: "all", limit: 20) {
    type
    score
    file {
      filename
      wordCount
      hasFrontMatter
    }
    schema {
      title
      description
    }
    highlight
  }
}
```

### Database Overview
Get a complete overview of your MarkiTect database:

```graphql
{
  databaseStats {
    totalFiles
    totalSchemas
    totalSizeBytes
    lastUpdated
  }
  recentFiles: markdownFiles(limit: 5) {
    filename
    createdAt
  }
  recentSchemas: schemas(limit: 5) {
    filename
    title
    createdAt
  }
}
```

## Integration Examples

### With JavaScript/TypeScript

```typescript
interface DatabaseStats {
  totalFiles: number;
  totalSchemas: number;
  totalSizeBytes: number;
  lastUpdated: string;
}

async function getStats(): Promise<DatabaseStats> {
  const response = await fetch('http://localhost:5000/graphql', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      query: `
        {
          databaseStats {
            totalFiles
            totalSchemas
            totalSizeBytes
            lastUpdated
          }
        }
      `
    })
  });

  const { data } = await response.json();
  return data.databaseStats;
}
```

### With Python

```python
import requests
from typing import List, Dict, Any

def search_content(query: str, limit: int = 10) -> List[Dict[str, Any]]:
    """Search for content in MarkiTect database."""

    response = requests.post('http://localhost:5000/graphql', json={
        'query': '''
        query SearchContent($query: String!, $limit: Int!) {
          search(query: $query, type: "all", limit: $limit) {
            type
            score
            file {
              filename
              wordCount
            }
            schema {
              title
              description
            }
            highlight
          }
        }
        ''',
        'variables': {
            'query': query,
            'limit': limit
        }
    })

    return response.json()['data']['search']

# Usage
results = search_content("API documentation", limit=5)
for result in results:
    print(f"Found {result['type']}: {result['highlight']}")
```

This GraphQL interface provides a powerful, flexible way to access and query MarkiTect data, enabling rich integrations and applications built on top of your Markdown content management system.