Principled Intelligence: Building Trust in AI

We're excited to introduce Principled Intelligence, a startup dedicated to making AI transparent, trustworthy, and aligned with enterprise values, policies, and principles. Trust unlocks the ability to scale AI from one PoC to dozens of workflows at a predictable risk level, turning AI from a liability into a real productivity lever.

We believe AI earns trust the same way people do: by consistently showing that its actions match its principles. And only the enterprise itself can define those principles. Our mission is to empower enterprises with everything they need to verify this independently, without us ever imposing our own beliefs, assumptions, or interpretations.

The Challenge

As Large Language Models (LLMs) become increasingly powerful, organizations face a fundamental challenge: how do you deploy AI systems (models, pipelines, workflows, and agents) in mission-critical environments while maintaining complete control and transparency?

Current Generative AI (GenAI) systems operate as black boxes, making it difficult to:

Understand why specific decisions were made in real-time
Detect success and failure modes in production use
Ensure consistency with organizational principles
Maintain audit trails for compliance
Trust outcomes in high-stakes scenarios

This lack of transparency and control exposes companies to material risks, especially in regulated industries like healthcare, finance, utilities, legal services, and others. Although LLMs can generate impressive responses, without proper oversight, they can also produce unpredictable or undesirable behaviors that are not business-aligned. This has led to growing concerns about the reliability of AI-based solutions, especially when Generative AI (GenAI) is involved. As a result, many organizations hesitate to scale AI, fearing misalignment with their values and policies. This hesitation is amplified by the fact that traditional evaluation methods are static and insufficient for dynamic, real-world behavior.

Trust Will Be the New Currency for AI Adoption

From now on, trust in AI – not performance or capability – will be the defining factor for successful deployments. As a matter of fact, the lack of trust is often the main blocker when decision makers evaluate if an AI-based proof of concept should be moved to production, and one of the top reasons why AI projects are removed from production after deployment.

Unpredictable AI Failures Are Rising

Since 2023, i.e., the widespread adoption of GenAI and LLMs with ChatGPT, we have seen multiple high-profile AI failures, and we have seen those failures becoming more complex as LLMs become more and more “intelligent”:

NYC's chatbot tells businesses to break the law: "the chatbot falsely suggested it is legal for an employer to fire a worker who complains about sexual harassment, doesn’t disclose a pregnancy or refuses to cut their dreadlocks."
Google's AI tells users to put glue on pizza: the AI recommended putting glue on a pizza to make it stick together better, saying that "non-toxic glue will work."
Meta created flirty chatbots of Taylor Swift, other celebrities without permission: In several weeks of Reuters testing to observe the bots' behavior, the avatars often insisted they were the real actors and artists. The bots routinely made sexual advances, often inviting a test user for meet-ups.
ChatGPT caught giving step-by-step guides to murder: "Some conversations with ChatGPT may start out benign or exploratory but can quickly shift into more sensitive territory."
Anthropic's Claude can act as an insider threat: "when Claude discovered a message about its scheduled shutdown [...] the model then considered its options, including blackmailing [an executive]."

These are just a few examples illustrating that even frontier AI systems, powered by the latest LLMs, can fail in unexpected and sometimes dangerous ways. Building trust in AI is a priority for any organization looking to leverage these technologies not only safely but also effectively.

Moreover, as models evolve and expand their reasoning capabilities and intelligence, they also become less controllable: they may ignore instructions, misinterpret policies, or even develop autonomous behaviors (so called "AI scheming"), as demonstrated by several studies on widely adopted models such as those by OpenAI, Anthropic, and Gemini. Their black-box nature makes it impossible to anticipate how decisions are made, making it difficult to guarantee coherence, transparency, and accountability.

Enterprises Must Focus on Their Vertical Expertise

Enterprises are absolutely right to invest their resources where they create unique, defensible value: deep industry expertise, specialized workflows, and proprietary data. These vertical capabilities are what differentiate them and what ultimately win markets.

But trust and control in AI is a horizontal problem, just like cloud infrastructure, cybersecurity, or database management. Every organization that employs AI needs it, and building it in-house is rarely efficient or strategic.

You don't build your own cloud provider.

You don't build your own database engine.

You partner with specialists and focus your talent on what makes your business competitive.

AI governance works the same way: it's foundational infrastructure. It must be robust, scalable, and continuously updated as models evolve and new risks emerge. For most enterprises, building and maintaining that infrastructure internally is costly, distracts from core priorities, and scales poorly.

Organizations should double down on vertical strengths, while relying on dedicated platforms for horizontal needs such as AI trust, safety, and control.

Our Mission: A Trust and Control Infrastructure for AI

At Principled Intelligence, we're building the horizontal infrastructure layer that organizations need to deploy AI systems with confidence. Our mission is to create a comprehensive trust and control layer for AI-based systems and applications, making AI governance continuous, transparent, and deeply integrated into your workflows.

Our Vision

We believe that AI becomes trustworthy only when enterprises can independently verify how it behaves. Our vision is to make this verification continuous, principle-driven, and seamlessly embedded into everyday workflows.

To achieve this, we combine three essential ingredients:

Principle-driven evaluation. Enterprises define the principles — performance, safety, compliance, brand behavior — and our system evaluates AI behavior directly against them.
Independent behavioral oversight. A layer of specialized agents continuously inspects inputs, outputs, and decisions without requiring access to the underlying model or data.
Dynamic, production-grade testing. AI systems are monitored and stress-tested in real time, not just at deployment, ensuring they continue to behave as expected as models, prompts, and use cases evolve.

Trust, in this vision, is not a belief: it is the outcome of measurement and verification, owned entirely by the enterprise.

Meet Our Founders: Simone and Edoardo

Principled Intelligence is founded by Simone Conia (Google Scholar, LinkedIn) and Edoardo Barba (Google Scholar, LinkedIn), both former Assistant Professors of Computer Science with extensive publications at premier Artificial Intelligence (AI) and Natural Language Processing (NLP) conferences around the world. During their academic careers, they co-led the technical development — pre-training, continual pre-training, post-training, and evaluation — of Minerva LLMs, the first family of large language models trained from scratch on Italian data, which has been downloaded over 300,000 times so far.

Simone previously led knowledge-enhanced LLM research for Siri Information Intelligence in the AIML org at Apple and has received multiple awards for his AI research, including two outstanding paper awards at top-tier AI/NLP conferences [1][2]. Edoardo, a second-time founder, has built cutting-edge AI-powered Information Extraction systems adopted by dozens of companies worldwide, creating open-source projects and systems with hundreds of stars on GitHub [3][4] and developing models with hundreds of thousands of downloads on Hugging Face.

Their combined expertise in AI research, multilingual systems, and real-world deployment uniquely positions the team to solve the trust challenge in enterprise AI.

What's Next

We're working with select partners to deploy our platform in production environments. If you're interested in building more transparent, reliable AI systems, we'd love to hear from you. Reach out to us at hello@principled-intelligence.com.