The Origins of Structure

Our world is pure chaos. It's an endless flow of unstructured data from everywhere: conversations, sights, experiences. The history of technological revolutions is the story of imposing structure on this chaos. Take the Industrial Revolution: it turned freeform manual labor into defined processes. People went from crafting entire products to pulling a lever or assembling a single part repeatedly.

The Internet did it once again. It took the messy, organic way humans interact and turned it into structured units: likes, comments, tweets. It reshaped human communication, reducing complex social exchanges into metrics and predefined actions. The beauty and utility of this transformation lie in its simplicity, even as it sacrifices nuance. By framing interactions in a structured, predictable format, the Internet made them programmable and scalable.

The Software Paradigm

Building modern software is the same idea: take the complexity of the real world and hammer it into rigid, specific use cases.

It’s not just about coding; it’s about abstraction. Every feature in an app or system represents a decision to carve out one sliver of reality and freeze it into logic. Behind every “save” button, there’s a set of assumptions about workflows, data types, and user behavior. The challenge of modern software development is deciding which parts of reality to model and which to ignore. This deliberate reduction is what makes systems useful.

But this approach has its limits. Real-world complexity often resists such reduction. Think of edge cases, inconsistencies, or unforeseen scenarios, things that don’t neatly fit into predefined boxes. This is where AI steps in, not to replace structured software but to extend its reach. AI can handle the ambiguous, the unstructured, and the irregular, the parts of reality traditional software couldn’t tame.

The Structured Data Revolution

The revolution AI is bringing isn’t about replacing these well-known processes. It’s about expanding them. AI transforms unstructured content into structured data and, crucially, back again. This isn’t a substitute for the way we’ve built technology; it’s the next layer. It’s about reaching places technology couldn’t touch before.

I’m incredibly optimistic about the future because this breaks down the barrier between “online” and “offline.” For decades, we’ve spent billions trying to drag businesses and people into the digital world. Now, we don’t have to; AI can bridge that gap for us.

The power to connect anything into the system is too powerful. That enables or scales a lot more possible business cases.

New Power

Let's try thinking about a couple of problems that before were impossible to scale or measure. Focusing on cases where we only need a human to understand the context, but now we can have a clue or at least prioritize before the human checks.

1. Do you want to measure the call metrics? With the meeting transcription, you can extract the key metrics that are important for your company.

       
         
   transcript = <<~TRANSCRIPT
   salesperson: Hi there! ...
   lead: Hello, my name is ...
   salesperson: Nice to meet you, ...
   ...
   TRANSCRIPT
   puts ActiveAI::DataExtractor.call(
     transcript,
     {
       salesperson_demonstrates_active_listening: {
         type: 'Integer',
         max: 10,
         min: 1
       },
       next_clear_step: {
         type: 'String',
         enum: ['Schedule a demo', 'Send a proposal', 'Follow-up in 3 days']
       }
     }
   )
   # { salesperson_demonstrates_active_listening: 8, next_clear_step: "Schedule a demo" }
             
       
     

2. Want to create an inventory without a system? Ok, take a photo every day of a paper with the notes.

       
         
   inventory_notes = File.open('inventory_notes.jpg')
   markdown = ActiveAI::ImageTranscribe.call(picture)
   puts ActiveAI::DataExtractor.call(
     markdown,
     { quantity_of_apples: { type: 'Integer', min: 0 } }
   ) # => { quantity_of_apples: 3 }
             
       
     

3. Do you have tons of 1:1 reports, and need to see all, but if prioritized?

       
         
   one_on_one_reports = [
     { score: nil, transcript: 'Overall, it\'s been all good...' },
     { score: nil, transcript: 'I noticed that you have been struggling...' },
     { score: nil, transcript: 'Last week I was on the beach...' },
   ]
   criteria = <<~CRITERIA
   Evaluate the internal NPS score based on the one-on-one report.
   Score 10 for positive and promoter employee and 0 for negative and detractor employee.
   CRITERIA
   one_on_one_reports.each_with_index do |report, index|
     one_on_one_reports[index][:score] = ActiveAI::Scorer.call(report, criteria)
   end
   
   prioritized_reports = one_on_one_reports.sort_by { |report| report[:score] }
             
       
     

4. Rank a list of shipping fees that shows first the cheapest or fastest? Ranking:

       
         
   shipping_fees = [
     { name: 'Standard', price: 0, days: 5 },
     { name: 'Express', price: 10, days: 2 },
   ]
   criteria = <<~CRITERIA
   Choose the shipping option, act like a real customer and make a fair guess.
   The shipping is for a pen priced at $1.
   CRITERIA
   ranked_items = ActiveAI::EloRanking.call(shipping_fees, criteria)
   puts ranked_items
   # => [{ name: 'Standard', score: 1600 }, { name: 'Express', score: 1000 }]
   
   criteria = <<~CRITERIA
   Choose the shipping option, act like a real customer and make a fair guess.
   The shipping is for a MacBook priced at $1000.
   CRITERIA
   ranked_items = ActiveAI::EloRanking.call(shipping_fees, criteria)
   puts ranked_items
   # => [{ name: 'Express', score: 1580 }, { name: 'Standard', score: 1020 }]
             
       
     

I can continue on for a lot more, but I think you understand. We gain a superpower for working with the abstract, creating determinism in places where it was not possible before. It is of course not perfect, and can hallucinate in some contexts. But this should not be a problem, the best place for GenAI is exactly when the answer is not explicitly clear.

Embrace the Flexibility

AI is going to eat the world, not because it’s intelligent, but because it’s flexible. We will generate and consume far more data from all places, even offline markets.

The revolution isn’t as glamorous as the media often portrays. Autonomous machines producing everything for us sounds amazing, but that’s not the reality. The transformer algorithm powering GenAI are, by design, incapable of being “intelligent” or alive.

What they can do is enormous: create structured data from anything. This is already generating a profound impact on society, and I believe it’s for the better. I want to be part of that, which is why I’m producing this kind of content and creating open-source libraries to help other engineers.

Welcome to the structured data revolution!