-
Traceability Matters: Gopal Shankar on Opening Up MySQL Development for the Next Decade
“Would making this capability available help a meaningful part of the MySQL community build better applications or run better systems?”
Q1. MySQL has recently made available several features from the enterprise tier to the MySQL Community Edition. What is driving this direction?
The simple answer is that Community Edition needs to stay strong for the people who build, run, and depend on MySQL every day. That includes developers, DBAs, startups, large enterprises, and software vendors.
When we bring broadly useful capabilities into Community Edition, particularly in areas like observability, high availability, performance, and developer experience, the whole MySQL ecosystem benefits. Users get a better platform, and we get better feedback from real workloads at scale.
Recent examples include replication observability and Group Replication capabilities, OpenTelemetry support, the Hypergraph Optimizer, Profile-Guided Optimization, and enhanced JSON Duality View support in Community Edition.
I see this as a clear sign of Oracle’s long-term commitment to MySQL. The Community Edition is fundamental to MySQL. Enterprise Edition continues to address additional commercial support, security, and operational needs, but the success of Community Edition is essential to the success of MySQL overall.
Q2. When Oracle makes a feature available in Community Edition, how are those decisions made?
There is not a single checklist that applies to every feature. We start with a practical question: would making this capability available help a meaningful part of the MySQL community build better applications or run better systems?
From there, we look at technical maturity, operational impact, security, compatibility, documentation, and long-term maintainability. A feature has to work well not only in a carefully controlled environment, but also in the many different environments where MySQL is deployed.
The strongest opportunities are often in areas that improve the daily experience of using MySQL: understanding what is happening in the server, operating highly available systems, diagnosing performance problems, and reducing unnecessary complexity for developers. We will continue to evaluate those opportunities as MySQL evolves.
Q3. What does MySQL do better than competitors in community engagement, and where can it improve?
MySQL’s strength is the combination of a mature open source database, deep engineering investment, and an ecosystem that has been built over decades. MySQL runs important workloads for organizations of every size, so reliability, compatibility, upgrades, tooling, and operational simplicity matter deeply to our community.
We also see opportunities to improve. The best open source communities make it easy for people to understand how an idea moves from discussion to action. We want to improve that path in MySQL through clearer design discussions, better issue triage, more visible contribution paths, and more predictable review and feedback.
Code is important, but it is not the only meaningful contribution. Testing, documentation, benchmarking, production feedback, tools, and community education all make MySQL better. We want contributors to feel that these forms of participation are recognized and useful.
Q4. MySQL 9.7.0 LTS brings capabilities previously limited to MySQL Enterprise Edition—including JSON Duality Views, the Hypergraph Optimizer, and improvements to replication observability and HA behavior—into MySQL Community Edition. Which of these changes do you think will have the greatest real-world impact for DBAs and developers, and what should teams consider before adopting them in production?
I would separate immediate operational impact from longer-term developer impact. For DBAs running Group Replication, the replication observability and HA changes will probably have the fastest and broadest benefit. Better visibility into flow control, applier lag and throughput, unhealthy members, and primary election helps teams diagnose problems earlier and make failover behavior more predictable. This is practical day-to-day value, especially for teams operating clusters at scale.
For developers, JSON Duality Views may be the more consequential change over time. They let teams work with JSON documents while retaining relational integrity and a single source of truth. The Hypergraph Optimizer can also be significant for complex queries, but its benefit will vary more by workload.
Teams should approach an LTS upgrade with thorough validation. Test representative workloads, failure scenarios, replication behavior, upgrades, and monitoring integrations in staging first. For the Hypergraph Optimizer, compare plans and performance for important queries. For JSON Duality Views, validate the data model, update paths, permissions, and concurrency behavior. And for telemetry, make sure the collector, retention policy, and handling of potentially sensitive operational data are ready before turning it on in production. The Community Edition additions cover replication and HA behavior, telemetry, JSON Duality Views, and the Hypergraph Optimizer.
Q5. You were personally involved in designing JSON Duality Views. What problem does it solve?
JSON Duality Views solve a problem many application teams know well. Developers often prefer JSON because it maps naturally to APIs and application objects. But relational modeling gives them normalization, transactional consistency, referential integrity, and SQL.
Historically, teams often had to choose one model, or build and maintain their own mapping layer between application objects and relational tables. In some cases, they also ended up duplicating data across multiple systems.
JSON Duality Views let an application work with hierarchical JSON documents while the underlying data remains relational. The application can use the model that feels natural for the task, but MySQL still provides a single source of truth.
For a team, that can mean less mapping code and simpler synchronization. It does not remove the need for good schema or API design, and it will not fit every application, but it gives suitable workloads a much simpler way to combine document-style development with relational strengths.
Q6. How do you balance new capabilities with MySQL’s simplicity and reliability?
MySQL has earned trust because it is practical. People can deploy it, operate it, upgrade it, and troubleshoot it with confidence. New capabilities must preserve that experience.
We pay close attention to defaults, configuration, backward compatibility, documentation, and operational behavior. Early Access builds, LTS releases, compatibility testing, and upgrade guidance are the practical mechanisms that help us validate that balance before broad adoption. The MySQL 9.6 foreign-key work is a good example. We moved foreign-key checks and cascades into the SQL layer so that those changes are visible to binary logs and CDC tools, while preserving compatibility, validating performance, and providing a temporary `innodb_native_foreign_keys` fallback for staged adoption. A feature should be powerful when users need it, while preserving the straightforward core MySQL experience for everyone else.
Not every user needs every new capability. Success is giving developers and DBAs useful new options while maintaining the stable, predictable MySQL experience that existing users rely on.
Q7. What does the more open community model look like in practice?
For a developer or DBA, participation should not begin only when they have a patch ready. They can discuss roadmap topics, share use cases, test Early Access releases, file actionable bugs, join GitHub discussions, contribute documentation or benchmarks, and participate in community events and contributor summits.
The important change is connecting these activities more clearly. If someone raises a good issue or proposal, they should be able to see where it goes next. Does it become a design discussion? A bug investigation? A request for testing? A roadmap input? That traceability matters.
Over the coming releases and community cycles, the community should see clearer guidance, more public technical discussion, improved GitHub workflows, and more structured ways to engage early. What matters most is whether people find the process easier to use and receive useful follow-up.
Q8. What changes are being made to improve the contributor experience, and how will you measure success?
The first improvement is clarity. Contributors need to know where to start, what information is needed, how review works, and what happens if a proposal is not accepted as submitted.
We are working toward clearer templates, better-defined contribution paths, more visible technical discussions, and stronger links between issues, proposals, patches, and bug records. This should make it easier for contributors and users to follow the progress of an idea.
The second improvement is feedback. When a contribution needs further refinement, people should receive a clear outcome, and where possible, practical guidance about what to do next. We will look at evidence: response and resolution trends, time to initial triage, review cycle time, contributor growth, contribution quality, roadmap participation, Early Access adoption, and feedback from contributors. We also intend to share progress regularly, pairing timely acknowledgment with meaningful follow-through.The goal is not simply to collect more pull requests. It is to create a community process that produces better outcomes.
Q9. MySQL recently marked 30 years. What will define the next decade?
MySQL must remain the practical, dependable choice for the application workloads that matter most: transactional systems, cloud-native services, distributed applications, and data-intensive workloads.
That means continued investment in performance, high availability, observability, security, developer productivity, and operational simplicity. These may not always be the most visible areas of innovation, but they are the reasons people trust a database in production.
The other important aspect is community participation. MySQL cannot thrive for another decade based only on work from one company. The community can influence priorities earlier, contribute effectively, build tools and extensions, share operational knowledge, and see that its feedback leads to visible action.
By the time MySQL turns 40, I would like it to be known not only for scale and reliability, but also for a community that has a real and practical role in shaping its future.
Qx. Anything else you wish to add?
I would encourage people to engage with MySQL early and directly. Try the Early Access releases, share concrete production experience, bring specific use cases, and tell us where the friction is.
The most useful feedback is grounded in real workloads and comes with enough detail for us to act on it. MySQL has always evolved through the combined work of engineers, users, customers, partners, and contributors. We want the next chapter to be even more collaborative.
………………………………………………………..
Gopal Shankar, Director of MySQL Engineering, Oracle.For over 20 years, I have worked at the heart of database engine architecture. Currently, as the Director of MySQL Engineering, I lead the organization responsible for the strategy, development, and roadmap of one of the world’s most popular database platforms. My expertise lies in the core internals of MySQL specifically kernel-level development, performance tuning, and scalability. I believe in solving complex technical challenges by prioritizing architectural simplicity and resilience. I have led the design of several important features, including the MySQL 8.0 Data Dictionary and Information Schema, as well as the recent JSON Duality feature. Additionally, I have helped architect the integration of foreign key handling directly into the SQL layer, effectively resolving long-standing trigger cascade limitations in MySQL recently. My goal is to deliver features that are high-performing, reliable and developer-friendly. I am focusing towards executing the MySQL roadmap and ensuring MySQL platform remains a powerful, solid foundation for modern applications. Beyond strategy, I stay connected to the kernel-level complexities tackling issues like high CPU usage, database corruption, and throughput bottlenecks. I am focused on fostering technical excellence and delivering a database engine that evolves with the needs of the industry.https://www.linkedin.com/in/gopal-shankar-1b34664/
………………….
Follow us on X
Follow us on LinkedIn
-
Vibe Coding a Database Lab: How I Built DBCanvas to Stop Rebuilding the Same Test Environment
Percona gives us room to work on our own AI-assisted projects, and I used mine to fix a problem I kept running into. Every time I wanted to try a new database feature, debug something tricky, or reproduce a customer issue, I ended up rebuilding much of the same infrastructure: DNS, TLS, Docker networking, database topologies, users and test data. Over the years, I wrote scripts to automate this but I still found myself copying and pasting post-installation steps from one lab to the next.
That repetition is what turned into DBCanvas, a self-hosted lab for designing, deploying, operating, and stress-testing multi-node database stacks on my own machine. Just design a topology on a canvas, click Deploy, and get real running nodes connected to the services and supporting infrastructure your test requires. Then, use the tools built into it or third party tools to load those databases, watch them work, and figure out why they’re misbehaving.
The code is up at github.com/jaimesicam/dbcanvas.
Figure 1: Deploying a multi-node MySQL topology with monitoring and orchestration
It’s vibe coded, and that’s the point
I want to be upfront about this that DBCanvas is vibe coded and built conversationally with an AI coding assistant rather than hand-written line by line. That turned out to be exactly the right approach for a tool whose whole job is to remove setup friction.
My initial loop or workflow looked like this. I started by asking the assistant to generate UI/UX demos for the frontend with React, a backend with Go, a drag-and-drop node canvas, a user management system and I iterated it until it felt right. Once I was satisfied, I asked it to turn those requirements into a SCAFFOLD.md which contained a full blueprint precise enough for the coding agent to rebuild the app from scratch as it contained the tech stack, naming conventions, directory tree, backend behavior, frontend behavior and the interactive details of the node editor itself.
Figure 2: The initial interface prototype that established the visual direction
Figure 3: The initial node-editor prototype for composing connected services
From there, I added features incrementally, budgeted by whatever tokens I had available in a session and logged every change in an IMPLEMENTATION.md so that I have a record of what it took to go from the original scaffold to wherever the project currently stood.
Figure 4: IMPLEMENTATION.md records each feature added after the initial scaffold
If I ever needed to rebuild the project from nothing, those two files are essentially the whole story. I know it is crude, but it’s my first time building with this many moving parts. It has held up so far at least for me.
The current loop looks like this:
Hit friction while testing, debugging, or learning something new.
Describe the environment that would remove that friction.
Let the assistant scaffold the automation: versions, configuration, identity, data, and tooling for that environment.
Keep whatever turned out to be reusable inside DBCanvas for next time.
Turn the whole workflow into something you can drive from a browser.
When Percona Server 8.4.11-11 shipped OpenID Connect authentication, I didn’t want to manually set up a Keycloak instance, wire up realms and clients, create sample identities, and configure Percona Server’s OIDC plugin every time I wanted to poke at it. Instead, that became a new DBCanvas setup. Deploy Keycloak + Percona Server + sample identities with a few clicks. You can inspect the generated OIDC configuration, authenticate with an ID token, and verify the mapped MySQL role, all without setting this up yourself. Vibe coding is what made it fast enough to build that scaffolding the same day the feature landed, and DBCanvas turned it into a reusable setup I can deploy again whenever I need it.
Figure 5: A Percona Server OIDC lab with Keycloak and a success mapped-role login
It’s not just about one feature… it’s a whole lab
OIDC with Keycloak is just one example. DBCanvas can also build a broader range of database environments:
MySQL: Percona XtraDB Cluster, Percona Server, MySQL Community, asynchronous replication, InnoDB Cluster and Group Replication, and MariaDB
PostgreSQL: standalone, Patroni, repmgr, Spock multi-master, CloudNativePG or Crunchy PGO on Kubernetes
MongoDB: Percona Server for MongoDB as standalone, replica set, or sharded cluster
Valkey: standalone and cluster
And around them, the infrastructure that makes a stack behave like a real environment. An Intranet node can provide DNS, mail, OpenLDAP, a Squid proxy and a certificate authority. Other nodes provide PMM, ProxySQL, HAProxy, Orchestrator, SeaweedFS S3, Keycloak, OpenBao, Samba AD DC, and a Kubernetes frame that runs any of six database operators.
A useful lab also needs activity. DBCanvas ships application simulators for a hotel booking system, an airline, a car rental fleet, and a stock exchange. Spin up a MongoDB replica set and point the stock market simulator at it and writes begin flowing through the set. You can observe elections and inspect real oplog activity instead of manually generating load against an idle cluster.
Figure 6: A MongoDB replica set running the stock market simulator with diagnostic tools attached
Leaning on tools other people already built
DBCanvas can make deployments more useful by integrating tools my colleagues have built for managing, troubleshooting and analyzing database environments. I can add them as part of the deployment workflow and place it alongside the database it is intended to work with.
MClusterAdmin, a MongoDB administration panel created by Przemek Malkowski, runs as its own node alongside a MongoDB deployment. It displays topology and replica-set status, sharding and the balancer, slow queries with explain, indexes, users and roles, all in a browser tab.
Figure 7: MClusterAdmin displaying the replica-set state for a DBCanvas deployment
Big Hole, an FTDC viewer, developed by Zelmar Michelini that decodes diagnostic.data. Drag in a folder and Big Hole charts every metric it can find. In DBCanvas, collecting the data is just as straightforward: right-click a MongoDB node, open the context menu, and download a compressed archive containing its logs and diagnostic data. Simply decompress the archive to your download directory, then drag the folder into Big Hole to visualize FTDC metrics alongside related log events.
Figure 8: Big Hole visualizing FTDC metrics downloaded from a MongoDB node
The same principle applies to deeper diagnostic work. DBCanvas automates the setup around proven tools instead of replacing them.
For example, the Operator Debugger uses Delve to step through Kubernetes operators with breakpoints, call stacks and variables.
Figure 9: Operator Debugger paused at a breakpoint inside a Percona operator
There’s also the Core Dump Analyzer where you can mount a core dump and matching binaries read-only, then inspect it with GDB.
Figure 10: Web-based Core Dump Analyzer with equivalent terminal command for manual troubleshooting
Ease of Use
While DBCanvas makes it easy to deploy environments but for troubleshooting, you still need to look deeper into the implementation within the nodes. Accessing the deployment via web terminal, regular terminal or web-based Filemanager would be helpful to have command of the deployment.
Figure 11: The file manager inspecting a MongoDB configuration inside a deployed node
The node menu provides the exact Docker exec command for direct access from a regular terminal.
Figure 12: Direct container access from a Docker exec command copied from the node menu
Testing beyond deployment
DBCanvas also comes with a data generator, a parallel query runner, a benchmark tool (OLTP/OLAP, read-write and read-only), and a packet inspector that decodes MySQL, PostgreSQL, MongoDB and Valkey traffic off the wire.
Figure 13: Data Generator creating sample rows for a deployed database
Figure 14: Packet Inspector decoding MongoDB traffic from the lab network
Together, these tools help reproduce problems facing a real deployment as I can generate data, apply load, inspect queries and examine network traffic.
Try it
bash
Copy
Copied!
git clone https://github.com/jaimesicam/dbcanvas.git && cd dbcanvas
make install
That builds the node images and starts DBCanvas at http://localhost:8080. The first run takes a while since it’s building docker images from scratch and learning which software versions are available for each OS.
Three important caveats:
DBCanvas creates disposable labs for previewing, testing and learning. It’s not a production software. It uses default credentials and favors setup speed over production security.
My workstation remains its primary test environment. If you run it elsewhere and encounter a bug or rough edge, please open a GitHub issue.
Some features are still maturing. I chose to bring many capabilities into one project, which means a few areas still need additional refinement and testing. For example, the Core Dump analyzer could benefit from more sophisticated variable extraction as well as the automatic detection of the appropriate operating system and compatible debug libraries to deploy.
If you’ve ever rebuilt the same test cluster for the third time this month, or wished you could hand a colleague a working reproduction instead of a page of setup instructions, try DBCanvas as it might save you some of the time it has saved me.
-
NodeJS MySQL Select Unique
I ran SELECT UNIQUE age FROM users on a table with a duplicated age. It returned 22, 17 and 15, the same rows SELECT DISTINCT returns. On MySQL 8 that statement stops before the table is read, because UNIQUE never entered the server’s select grammar, and DISTINCT is the modifier both servers accept. Which unique […]
-
Vector search in MySQL: an early look at HNSW and custom indexes in VillageSQL
MySQL 9.x added the VECTOR data type so you can store an embedding. You get a VECTOR column, functions to convert to and from text, and VECTOR_DIM() to ask how wide a vector is.
What you don't get is a way to compare two vectors. The 9.x community server has no distance function and no vector index. MySQL 8.4 has none of this at all. You can't add the missing index yourself either because vanilla MySQL's list of index types is fixed to B-tree, R-tree, hash, and full-text.
That's where VillageSQL comes in. VillageSQL is the innovation platform for MySQL that adds an extension framework (similar to PostgreSQL's extension framework) to enable permissionless innovation. Instead of waiting for a feature to be implemented in a few years in a future version of MySQL, new functionality can be dynamically added to a version of MySQL you run today.
VillageSQL already supports custom functions and custom data types, and we've been working on custom indexes. With custom indexes, an extension can define a whole index type including how it's stored in InnoDB, how it's built as rows arrive, and how it's searched. The server treats the custom index as a first-class index, the same as the built-in index types. The server even plans queries against the custom index and reports it by name.
Vector search is a key feature for AI-era applications, so that's the first use case for custom indexes we are building.
At Percona Live in Amsterdam on September 9, 2026, we demoed an early build of this vector work (not yet merged). Below is the annotated demo.
Start with a server
The demo starts by installing VillageSQL. You copy one line from the website, paste it into a terminal, and you have a prebuilt server.
curl -fsSL https://install.villagesql.com | bash
That gets you a stable server. Please note the vector work highlighted below runs on an unmerged, development build.
Declaring a vector index
With the server up, the demo installs the vsql-vector extension with an ordinary INSTALL EXTENSION, and then creates one table:
CREATE TABLE demo_vectors (
id INT PRIMARY KEY,
embedding SVECTOR(3) NOT NULL,
INDEX idx_embedding (embedding hnsw_l2) USING EXTENDED(hnsw)
) ENGINE=InnoDB;
USING EXTENDED(hnsw) names a custom index type and SVECTOR is a custom column type. The extension supplies both. The server stores the metadata, plans the query, and hands the work off. Once the table exists, the vectors arrive by plain INSERT.
Does the server use it?
You can ask the server whether it will really use that index:
EXPLAIN FORMAT=TREE SELECT id FROM demo_vectors
ORDER BY l2_distance(embedding, '[1.0,2.0,3.0]') LIMIT 3\G
The plan comes back as Custom index distance scan on idx_embedding, which is the server saying it would answer from the HNSW graph instead of reading every row. Running that same SELECT without EXPLAIN in front of it returns the three nearest rows, closest first.
Fast and correct
The demo then builds two tables that hold the same 20,000 vectors. One table has an HNSW index on the vector column and the other has no index at all, and the same nearest-neighbor query runs against both.
Across 50 queries each, the indexed table averages 0.19 ms and the full scan 3.82 ms, which is 20x slower. This is synthetic data at 3 dimensions, so read it as a mechanism check rather than a benchmark.
Speed alone does not prove an index is beneficial, because an approximate index can be fast by being wrong. The unindexed table holds the same vectors, so it gives the exact answer to compare against. Here the index returns nine of the true top ten, and the tenth row is the price of an approximate search. ef_search is a dial that allows you to tune accuracy and performance.
Where it lands on real embeddings
We ran ann-benchmarks against fashion-mnist-784 on an 8 vCPU Xeon VM. Below are our initial results. When vector indexes are merged, we’ll publish steps to reproduce these benchmark results.
fashion-mnist-784 — 60,000 vectors, k=10, 8 vCPU Xeon
ef_search
queries/sec
recall@10
50
635
99.68%
100
440
99.85%
200
290
99.92%
These numbers are a snapshot of the work which is still in progress. They come from one early build, and we expect them to move. The shape of the curve is the interesting part. Each step up in ef_search buys recall and costs throughput. Going from 50 to 100 adds 0.17 points of recall and gives up about 30% of the queries per second. Going to 200 adds another 0.07 points and costs about half. You can choose the point on that curve that your application needs.
How it fits together
The server treats the extension's index as a real one. Ask what indexes a table has, and the vector ones come back as HNSW, sitting next to an ordinary BTREE. SHOW CREATE TABLE (from a different example table with two vector columns) returns the full definition:
KEY `idx_title` (`title_vec` `vsql_vector`.`hnsw_cosine`)
USING EXTENDED(`vsql_vector`.`hnsw`) WITH (`ef_construction` = 64, `m` = 8),
KEY `idx_body` (`body_vec` `vsql_vector`.`hnsw_l1`),
KEY `idx_tag` (`tag`)
M and ef_construction are the extension's own build knobs. The server stores them without knowing what they mean, and prints them back in DDL you can replay. The vectors themselves sit in InnoDB's own storage, so there's no shadow table and no second write on insert. Columns go up to 3,072 dimensions (for comparison, pgvector supports 2,000 dimensions for an indexed column), and one table can carry several vector indexes, each on its own column with its own distance function.
What's next
Filtered search, deletes, wider version coverage, and comparison benchmarks are all still ahead. We expect to deliver the custom index framework for both 8.4 and 9.7 codebases.
This is an initial build of unreleased work, and we wanted to show it while it's still moving. If you want to read the code in the meantime, the extension is at github.com/villagesql/vsql-vector, and you can get the server from villagesql.com. VillageSQL Server supports MySQL 8.4, 9.7, and Percona Server 8.4.
Please let us know your feedback. You can find us on Discord or on GitHub Issues.
-
Node.js MySQL Insert Record
I pointed a Node script at a MySQL table and the connection succeeded. The INSERT failed on a name containing an apostrophe. Pasting values into the SQL string holds until real data arrives, then a name like O’Brien ends the run with a syntax error. Placeholders are the fix, and the driver escapes every value […]
|
