Last week in Amsterdam, 52 CTOs, engineering leaders and AI leads came together to discuss a challenge that is becoming increasingly urgent: how to stay in control of AI without slowing innovation. AI adoption is moving quickly. In many organisations, it is moving faster than anyone has a complete view of what is being built, bought or used.
That tension set the agenda for the latest Gapstars CTO Club session in Amsterdam. Fifty-two CTOs, engineering leaders and AI leads joined Gapstars and Deeploy to explore a question that is becoming difficult to postpone:
How do you stay in control of AI without slowing down the teams building with it?
Deeploy CEO Maarten Stolk and Lead Engineer Robbert van der Gugten led the session. But much of its value came from the questions, challenges and experiences shared by the technology leaders in the room.
The discussion pointed to a practical conclusion: AI governance is no longer only a compliance topic. It is becoming part of how modern engineering organisations operate.
You can only govern what you can see
AI is appearing across organisations in ways that are easy to miss. A sales team may use an agent for company research. HR may experiment with AI-supported screening. Developers introduce coding assistants. Product teams connect models to internal data and build their own agents.
The issue is not necessarily that these experiments are happening. The issue is not knowing they are happening.
That is why visibility comes first in Deeploy’s framework. Before designing a complex governance programme, create a shared view of the AI systems and agents already in use: what each one does, who owns it, where it sits in the lifecycle and what risk it may introduce.
A central AI registry turns an abstract governance discussion into something teams can act on. It also exposes the gaps that matter most: systems without owners, unclear data use, undocumented vendor tools and experiments that are already closer to production than anyone realised.
Not every AI system needs the same controls
Once you know what is running, the next question is risk. A tool that helps a salesperson research prospective customers is fundamentally different from a system that influences credit decisions or processes sensitive personal information.
Treating both in exactly the same way creates one of two problems: unnecessary bureaucracy for low-risk use cases, or insufficient control for high-impact ones.
A risk-based approach starts with a small set of practical questions:
- What is the system intended to do?
- Who will use it – and who could be affected by it?
- How could it be misused?
- What happens if it gets something wrong?
- Could the impact be difficult or impossible to reverse?
The answers determine the level of scrutiny, the controls that apply and the points at which a system should be reviewed. In practice, that requires engineering, product, security and compliance to work much more closely together.
Governance has to become part of engineering
For the technology leaders present at our Gapstars office in Amsterdam, this was perhaps the most important takeaway of the evening. Governance works poorly when it becomes another approval process sitting outside the engineering team.
A better approach is to build controls into the way AI systems are developed and operated: define what must be documented, when a review is required, how behaviour is evaluated and what happens after a system reaches production.
Predictive models may call for familiar measures such as performance and drift monitoring. Generative AI and agents introduce different questions. Teams need realistic test sets, traces of agent behaviour, monitoring of tool calls and alerts for unexpected activity. Evaluation cannot stop at launch; production behaviour needs to be compared with what teams observed during testing.
Where the potential impact is too high, guardrails should prevent certain inputs, outputs or actions – not simply report them afterwards. Governance then becomes part of the engineering lifecycle, rather than a document produced at the end of it.
Human oversight still matters – when it is designed well
Adding a human approval step to everything is not scalable. Removing people entirely from important decisions creates problems of its own. The real design question is where human judgement genuinely changes the outcome.
For high-impact or irreversible actions – such as making a payment, sending a sensitive communication or changing a critical record – approval before action may be appropriate. Elsewhere, sampled review of production activity may provide better oversight without creating a queue around every decision.
Context matters too. A reviewer cannot meaningfully assess an AI output without the relevant references, trace or explanation. A generic ‘human in the loop’ requirement is not a control unless the person has enough information and authority to intervene.
Five no-regret moves
Deeploy brought the discussion back to five practical moves. Together, they offer a sensible starting point without requiring one large governance programme.
- Classify AI systems by risk
Use intended purpose, users, potential misuse and impact to decide how much control each use case needs.
- Create an AI registry
Record every internal and third-party AI system, with a clear owner, risk level and lifecycle stage.
- Define a control framework
Make controls conditional on risk, lifecycle stage and your organisation’s role as provider or deployer.
- Evaluate, monitor and alert
Test realistic behaviour before launch, observe what happens in production and define signals that require intervention.
- Add guardrails where needed
Restrict risky inputs, outputs and actions, then evaluate the guardrails themselves and monitor how they behave.
None of these moves solves AI governance on its own. Together, they create visibility, ownership and feedback loops – the foundations organisations need before adding more policy or tooling.
Why conversations like this matter
There is rarely one perfect answer to questions this new. Technology leaders are working through many of the same challenges: how to adopt AI responsibly, how to scale engineering teams, how to introduce useful processes without unnecessary complexity and how the role of engineering leadership must evolve.
Bringing those leaders together creates a different conversation from a conference presentation or another whitepaper. People compare approaches, challenge assumptions and leave with ideas they can take back to their teams.
That is the role Gapstars wants the CTO Club to play: a place where peers can examine the realities behind modern technology leadership. It also connects closely to how we work with clients. Building strong technology teams is not only about adding engineering capacity; it is about creating teams that can adapt as the practice of building software changes.
AI governance is quickly becoming part of that equation.
Join the next conversation
This was one session in an ongoing series. Gapstars CTO Club brings together tech leaders from Benelux, the UK, and the US for summits, peer networking, and sessions like this one throughout the year.
Want the regulatory detail in full? Deeploy’s whitepaper on the AI Governance Control Framework goes deeper into the control framework, EU AI Act timelines, and maturity model referenced above: deeploy.ai/white-paper-ai-governance-control-framework.
Join us for the next session. Contact Joran de Vries, our CTO Community Lead, at [email protected], or register to be part of our community at gapstars.net/tech/cto-club.
