> ## Documentation Index
> Fetch the complete documentation index at: https://ricardovelit.com/axon-docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Extraction

> Extract structured data from unstructured text with validation and confidence tracking

## Overview

This example demonstrates how to extract structured data from unstructured documents using AXON's `probe` operation, epistemic types, and validation. It shows how to handle missing fields, ensure data quality, and maintain confidence scores.

## Use Case

Extract structured information from:

* Resumes and CVs
* Invoices and receipts
* Product descriptions
* News articles
* Customer feedback
* Research papers

## Complete Code

```axon data_extraction.axon theme={null}
// AXON Example — Data Extraction
// Extract structured data with validation and confidence tracking

persona DataExtractor {
  domain: ["information extraction", "NLP", "data processing"]
  tone: precise
  confidence_threshold: 0.80
}

context ExtractionMode {
  memory: none
  language: "en"
  depth: thorough
  max_tokens: 2048
  temperature: 0.1
}

anchor NoGuessing {
  require: text_evidence
  confidence_floor: 0.75
  unknown_response: "Field not found in source text"
  on_violation: raise AnchorBreachError
}

type Email where matches(pattern: email_regex)

type PhoneNumber where matches(pattern: phone_regex)

type Currency(0.0..1000000.0)

type Person {
  name: FactualClaim,
  email: Email?,
  phone: PhoneNumber?,
  location: FactualClaim?
}

type Experience {
  company: FactualClaim,
  role: FactualClaim,
  duration: FactualClaim,
  description: Opinion?
}

type ResumeData {
  person: Person,
  experiences: List<Experience>,
  skills: List<FactualClaim>,
  education: List<FactualClaim>,
  confidence: ConfidenceScore
}

flow ExtractResume(doc: Document) -> ResumeData {
  step ExtractPerson {
    given: doc
    probe doc for [name, email, phone, location]
    output: Person
  }
  
  validate ExtractPerson.output against PersonSchema {
    if confidence < 0.80 -> refine(max_attempts: 2)
    if name == ∅ -> raise MissingRequiredFieldError
  }
  
  step ExtractExperience {
    given: doc
    ask: "Extract all work experience entries with company, role, and duration"
    output: List<Experience>
  }
  
  step ExtractSkills {
    given: doc
    probe doc for [skills, technologies, certifications]
    output: SkillMap
  }
  
  step ExtractEducation {
    given: doc
    ask: "Extract educational background"
    output: List<FactualClaim>
  }
  
  weave [
    ExtractPerson.output,
    ExtractExperience.output,
    ExtractSkills.output,
    ExtractEducation.output
  ] into ResumeData {
    format: StructuredReport
  }
}

run ExtractResume(resumeDoc)
  as DataExtractor
  within ExtractionMode
  constrained_by [NoGuessing]
  on_failure: retry(backoff: linear)
  output_to: "extracted.json"
  effort: medium
```

## Key Components

### Persona: DataExtractor

```axon theme={null}
persona DataExtractor {
  domain: ["information extraction", "NLP", "data processing"]
  tone: precise
  confidence_threshold: 0.80
}
```

Defines an extraction specialist:

* **Domain**: Information extraction, NLP, data processing
* **Tone**: Precise (exact, no embellishment)
* **High threshold**: 0.80 for accurate extraction

### Context: ExtractionMode

```axon theme={null}
context ExtractionMode {
  memory: none
  language: "en"
  depth: thorough
  max_tokens: 2048
  temperature: 0.1
}
```

Configured for extraction:

* **Stateless**: No memory (each extraction independent)
* **Thorough**: Careful examination
* **Very low temperature**: 0.1 for deterministic extraction

### Anchor: NoGuessing

```axon theme={null}
anchor NoGuessing {
  require: text_evidence
  confidence_floor: 0.75
  unknown_response: "Field not found in source text"
  on_violation: raise AnchorBreachError
}
```

Prevents hallucination:

* **Requires**: Evidence from source text
* **Minimum confidence**: 0.75
* **Explicit unknowns**: "Field not found" instead of guessing

<Warning>
  Never let the LLM "fill in" missing fields with plausible guesses. Use the `NoGuessing` anchor to ensure all extractions cite source text.
</Warning>

### Custom Types with Validation

```axon theme={null}
type Email where matches(pattern: email_regex)
type PhoneNumber where matches(pattern: phone_regex)
```

Refinement types with pattern matching:

* Compile-time guarantee of format
* Runtime validation

```axon theme={null}
type Currency(0.0..1000000.0)
```

Range-constrained for monetary values.

```axon theme={null}
type Person {
  name: FactualClaim,
  email: Email?,
  phone: PhoneNumber?,
  location: FactualClaim?
}
```

Structured person data:

* `name`: Required factual claim
* `email`, `phone`, `location`: Optional validated fields

```axon theme={null}
type Experience {
  company: FactualClaim,
  role: FactualClaim,
  duration: FactualClaim,
  description: Opinion?
}
```

Work experience entry:

* Company, role, duration: Facts
* Description: Opinion (subjective characterization)

### Flow: ExtractResume

Four-step extraction pipeline:

**Step 1: ExtractPerson**

```axon theme={null}
step ExtractPerson {
  given: doc
  probe doc for [name, email, phone, location]
  output: Person
}
```

Uses `probe` for targeted field extraction.

**Validation**

```axon theme={null}
validate ExtractPerson.output against PersonSchema {
  if confidence < 0.80 -> refine(max_attempts: 2)
  if name == ∅ -> raise MissingRequiredFieldError
}
```

Ensures:

* High confidence (≥0.80)
* Required field (name) present

**Step 2: ExtractExperience**

```axon theme={null}
step ExtractExperience {
  given: doc
  ask: "Extract all work experience entries with company, role, and duration"
  output: List<Experience>
}
```

Extracts multiple experience entries.

**Step 3: ExtractSkills**

```axon theme={null}
step ExtractSkills {
  given: doc
  probe doc for [skills, technologies, certifications]
  output: SkillMap
}
```

Probes for skills-related fields.

**Step 4: ExtractEducation**

```axon theme={null}
step ExtractEducation {
  given: doc
  ask: "Extract educational background"
  output: List<FactualClaim>
}
```

Extracts education as list of facts.

**Synthesis**

```axon theme={null}
weave [
  ExtractPerson.output,
  ExtractExperience.output,
  ExtractSkills.output,
  ExtractEducation.output
] into ResumeData {
  format: StructuredReport
}
```

Combines all extractions into structured output.

## Usage

### Run Extraction

```bash theme={null}
# Validate
axon check data_extraction.axon

# Compile
axon compile data_extraction.axon

# Execute
axon run data_extraction.axon --backend anthropic --trace
```

### Example Input (Resume)

```text theme={null}
John Smith
Senior Software Engineer
john.smith@email.com | (555) 123-4567 | San Francisco, CA

EXPERIENCE

TechCorp Inc. | Senior Software Engineer | 2020 - Present
- Led team of 5 engineers building cloud infrastructure
- Designed and implemented microservices architecture
- Reduced deployment time by 60%

StartupXYZ | Software Engineer | 2018 - 2020
- Developed full-stack web applications using React and Node.js
- Implemented CI/CD pipelines

SKILLS
Python, JavaScript, React, Node.js, Docker, Kubernetes, AWS, PostgreSQL

EDUCATION
B.S. Computer Science, Stanford University, 2018
```

### Example Output

```json theme={null}
{
  "type": "ResumeData",
  "person": {
    "name": "John Smith",
    "email": "john.smith@email.com",
    "phone": "(555) 123-4567",
    "location": "San Francisco, CA"
  },
  "experiences": [
    {
      "company": "TechCorp Inc.",
      "role": "Senior Software Engineer",
      "duration": "2020 - Present",
      "description": "Led team of 5 engineers building cloud infrastructure"
    },
    {
      "company": "StartupXYZ",
      "role": "Software Engineer",
      "duration": "2018 - 2020",
      "description": "Developed full-stack web applications"
    }
  ],
  "skills": [
    "Python",
    "JavaScript",
    "React",
    "Node.js",
    "Docker",
    "Kubernetes",
    "AWS",
    "PostgreSQL"
  ],
  "education": [
    "B.S. Computer Science, Stanford University, 2018"
  ],
  "confidence": 0.92
}
```

## Advanced Patterns

### Invoice Extraction

```axon theme={null}
type Invoice {
  invoice_number: FactualClaim,
  date: FactualClaim,
  vendor: FactualClaim,
  total: Currency,
  items: List<LineItem>,
  confidence: ConfidenceScore
}

type LineItem {
  description: FactualClaim,
  quantity: Integer,
  price: Currency,
  total: Currency
}

flow ExtractInvoice(doc: Document) -> Invoice {
  step ExtractHeader {
    given: doc
    probe doc for [invoice_number, date, vendor, total]
    output: InvoiceHeader
  }
  
  step ExtractLineItems {
    given: doc
    ask: "Extract all line items with description, quantity, and price"
    output: List<LineItem>
  }
  
  validate ExtractLineItems.output against LineItemSchema {
    if any_price < 0.0 -> raise InvalidDataError
    if sum(items.total) != header.total -> warn "Total mismatch"
  }
  
  weave [ExtractHeader.output, ExtractLineItems.output] into Invoice
}
```

### Product Data Extraction

```axon theme={null}
type Product {
  name: FactualClaim,
  price: Currency,
  description: FactualClaim,
  features: List<FactualClaim>,
  reviews: Opinion?,
  availability: FactualClaim
}

flow ExtractProduct(doc: Document) -> Product {
  step ExtractBasics {
    given: doc
    probe doc for [name, price, description, availability]
    output: ProductBasics
  }
  
  step ExtractFeatures {
    given: doc
    ask: "List all product features and specifications"
    output: List<FactualClaim>
  }
  
  step ExtractReviews {
    given: doc
    ask: "Summarize customer reviews and opinions"
    output: Opinion
  }
  
  weave [ProductBasics, ExtractFeatures.output, ExtractReviews.output] into Product
}
```

### Multi-Document Extraction

```axon theme={null}
flow ExtractBatch(docs: List<Document>) -> List<ResumeData> {
  step ExtractAll {
    given: docs
    ask: "Extract resume data from each document"
    output: List<ResumeData>
  }
  
  validate ExtractAll.output against BatchSchema {
    if any_confidence < 0.75 -> refine(max_attempts: 1)
  }
}
```

### Incremental Extraction with Memory

```axon theme={null}
context IncrementalMode {
  memory: session
  language: "en"
  depth: thorough
}

flow IncrementalExtract(doc: Document) -> ExtractedData {
  recall("previous extractions") from SessionMemory
  
  step Extract {
    given: [doc, PreviousExtractions]
    ask: "Extract new information, avoiding duplicates"
    output: NewData
  }
  
  remember(NewData) -> SessionMemory
}
```

## Best Practices

### 1. Use Probe for Targeted Extraction

```axon theme={null}
// ✅ Good: Targeted field extraction
step Extract {
  probe doc for [name, email, phone]
  output: Person
}

// ❌ Less efficient: Open-ended
step Extract {
  ask: "Extract person information"
  output: Person
}
```

### 2. Validate Required Fields

```axon theme={null}
validate Person against PersonSchema {
  if name == ∅ -> raise MissingRequiredFieldError
  if email == ∅ -> warn "Email not found"
}
```

### 3. Use Optional Types for Missing Data

```axon theme={null}
type Person {
  name: FactualClaim,      // Required
  email: Email?,           // Optional
  phone: PhoneNumber?      // Optional
}
```

### 4. Apply Range Constraints

```axon theme={null}
type Price(0.0..1000000.0)  // Prevent negative or absurd prices
type Quantity(1..10000)     // Reasonable quantity range
```

### 5. Use Very Low Temperature

```axon theme={null}
context ExtractionMode {
  temperature: 0.1  // Deterministic extraction
}
```

### 6. Require Text Evidence

```axon theme={null}
anchor NoGuessing {
  require: text_evidence
  unknown_response: "Field not found"
}
```

## Related Examples

<CardGroup cols={2}>
  <Card title="Contract Analyzer" icon="file-contract" href="/axon-docs/axon-docs/examples/contract-analyzer">
    Legal contract analysis with entity extraction
  </Card>

  <Card title="Sentiment Analysis" icon="face-smile" href="/axon-docs/axon-docs/examples/sentiment-analysis">
    Analyze text sentiment with confidence tracking
  </Card>

  <Card title="Multi-Step Reasoning" icon="brain" href="/axon-docs/axon-docs/examples/multi-step-reasoning">
    Complex reasoning with chain-of-thought
  </Card>
</CardGroup>

## Related Documentation

* [Flow](/axon-docs/axon-docs/language/flow) — Probe and extraction operations
* [Types](/axon-docs/axon-docs/language/types) — Refinement types and validation
* [Anchor](/axon-docs/axon-docs/language/anchor) — Prevent hallucination
* [Persona](/axon-docs/axon-docs/language/persona) — Define extraction specialists