Close Menu
    Machine Learning & Research

    Newbie’s Information to Information Extraction with LangExtract and LLMs

    Oliver ChambersBy No Comments6 Mins Read
    Newbie’s Information to Information Extraction with LangExtract and LLMs


    Newbie’s Information to Information Extraction with LangExtract and LLMs
    Picture by Creator

     

    # Introduction

     
    Do you know that a big portion of helpful data nonetheless exists in unstructured textual content? For instance, analysis papers, medical notes, monetary stories, and so on. Extracting dependable and structured data from these texts has at all times been a problem. LangExtract is an open-source Python library (launched by Google) that solves this downside utilizing giant language fashions (LLMs). You possibly can outline what to extract through easy prompts and some examples, after which it makes use of LLMs (like Google’s Gemini, OpenAI, or native fashions) to drag out that data from paperwork of any size. One other factor that makes it helpful is its help for very lengthy paperwork (by chunking and multi-pass processing) and interactive visualization of outcomes. Let’s discover this library in additional element.

     

    # 1. Putting in and Setting Up

     
    To put in LangExtract domestically, first guarantee you might have Python 3.10+ put in. The library is accessible on PyPI. In a terminal or digital setting, run:

     

    For an remoted setting, chances are you’ll first create and activate a digital setting:

    python -m venv langextract_env
supply langextract_env/bin/activate  # On Home windows: .langextract_envScriptsactivate
pip set up langextract

     

    There are different choices from supply and utilizing Docker as nicely you could examine from right here.

     

    # 2. Setting Up API Keys (for Cloud Fashions)

     
    LangExtract itself is free and open-source, however when you use cloud-hosted LLMs (like Google Gemini or OpenAI GPT fashions), you should provide an API key. You possibly can set the LANGEXTRACT_API_KEY setting variable or retailer it in a .env file in your working listing. For instance:

    export LANGEXTRACT_API_KEY="YOUR_API_KEY_HERE"

     
    or in a .env file:

    cat >> .env << 'EOF'
LANGEXTRACT_API_KEY=your-api-key-here
EOF
echo '.env' >> .gitignore

     
    On-device LLMs through Ollama or different native backends don’t require an API key. To allow OpenAI, you’ll run pip set up langextract[openai], set your OPENAI_API_KEY, and use an OpenAI model_id. For Vertex AI (enterprise customers), service account authentication is supported.

     

    # 3. Defining an Extraction Activity

     
    LangExtract works by you telling it what data to extract. You do that by writing a transparent immediate description and supplying a number of ExampleData annotations that present what an accurate extraction appears to be like like on pattern textual content. As an illustration, to extract characters, feelings, and relationships from a line of literature, you may write:

    import langextract as lx

immediate = """
  Extract characters, feelings, and relationships so as of look.
  Use actual textual content for extractions. Don't paraphrase or overlap entities.
  Present significant attributes for every entity so as to add context."""
examples = [
    lx.data.ExampleData(
        text="ROMEO. But soft! What light through yonder window breaks? ...",
        extractions=[
            lx.data.Extraction(
                extraction_class="character",
                extraction_text="ROMEO",
                attributes={"emotional_state": "wonder"}
            ),
            lx.data.Extraction(
                extraction_class="emotion",
                extraction_text="But soft!",
                attributes={"feeling": "gentle awe"}
            )
        ]
    )
]

     
    These examples (taken from LangExtract’s README) inform the mannequin precisely what sort of structured output is predicted. You possibly can create related examples to your area.

     

    # 4. Working the Extraction

     
    As soon as your immediate and examples are outlined, you merely name the lx.extract() perform. The important thing arguments are:

    • text_or_documents: Your enter textual content, or an inventory of texts, or perhaps a URL string (LangExtract can fetch and course of textual content from a Gutenberg or different URL).
    • prompt_description: The extraction directions (a string).
    • examples: A listing of ExampleData that illustrate the specified output.
    • model_id: The identifier of the LLM to make use of (e.g. "gemini-2.5-flash" for Google Gemini Flash, or an Ollama mannequin like "gemma2:2b", or an OpenAI mannequin like "gpt-4o").
    • Different non-obligatory parameters: extraction_passes (to re-run extraction for increased recall on lengthy texts), max_workers (to do parallel processing on chunks), fence_output, use_schema_constraints, and so on.

    For instance:

    input_text=""'JULIET. O Romeo, Romeo! wherefore artwork thou Romeo?
Deny thy father and refuse thy title;
Or, if thou wilt not, be however sworn my love,
And I am going to not be a Capulet.
ROMEO. Shall I hear extra, or shall I communicate at this?
JULIET. 'Tis however thy title that's my enemy;
Thou artwork thyself, although not a Montague.
What’s in a reputation? That which we name a rose
By every other title would odor as candy.'''


consequence = lx.extract(
    text_or_documents=input_text,
    prompt_description=immediate,
    examples=examples,
    model_id="gemini-2.5-flash"
)

     
    This sends the immediate and examples together with the textual content to the chosen LLM and returns a Outcome object. LangExtract robotically handles tokenizing lengthy texts into chunks, batching calls in parallel, and merging the outputs.

     

    # 5. Dealing with Output and Visualization

     
    The output of lx.extract() is a Python object (usually referred to as consequence) that accommodates the extracted entities and attributes. You possibly can examine it programmatically or put it aside for later. LangExtract additionally gives helper features to save lots of outcomes: for instance, you possibly can write the outcomes to a JSONL (JSON Strains) file (one doc per line) and generate an interactive HTML assessment. For instance:

    lx.io.save_annotated_documents([result], output_name="extraction_results.jsonl", output_dir=".")
html = lx.visualize("extraction_results.jsonl")
with open("viz.html", "w") as f:
    f.write(html if isinstance(html, str) else html.knowledge)

     
    This writes an extraction_results.jsonl file and an interactive viz.html file. The JSONL format is handy for big datasets and additional processing, and the HTML file highlights every extracted span in context (color-coded by class) for simple human inspection like this:
     
    Output and Visualization: LangextractOutput and Visualization: Langextract
     

    # 6. Supporting Enter Codecs

     
    LangExtract is versatile about enter. You possibly can provide:

    • Plain textual content strings: Any textual content you load into Python (e.g. from a file or database) could be processed.
    • URLs: As proven above, you possibly can go a URL (e.g. a Venture Gutenberg hyperlink) as text_or_documents="https://www.gutenberg.org/information/1513/1513-0.txt". LangExtract will obtain and extract from that doc.
    • Record of texts: Go a Python record of strings to course of a number of paperwork in a single name.
    • Wealthy textual content or Markdown: Since LangExtract works on the textual content stage, you may additionally feed in Markdown or HTML when you pre-process it to uncooked textual content. (LangExtract itself doesn’t parse PDFs or photos, it is advisable to extract textual content first.)

     

    # 7. Conclusion

     
    LangExtract makes it straightforward to show unstructured textual content into structured knowledge. With excessive accuracy, clear supply mapping, and easy customization, it really works nicely when rule-based strategies fall quick. It’s particularly helpful for advanced or domain-specific extractions. Whereas there’s room for enchancment, LangExtract is already a powerful instrument for extracting grounded data in 2025.
     
     

    Kanwal Mehreen is a machine studying engineer and a technical author with a profound ardour for knowledge science and the intersection of AI with medication. She co-authored the book “Maximizing Productiveness with ChatGPT”. As a Google Technology Scholar 2022 for APAC, she champions range and educational excellence. She’s additionally acknowledged as a Teradata Variety in Tech Scholar, Mitacs Globalink Analysis Scholar, and Harvard WeCode Scholar. Kanwal is an ardent advocate for change, having based FEMCodes to empower girls in STEM fields.

    Share.

    Related Posts