openark.orgcode.openark.org – Blog by Shlomi Noach

Skip to content code.openark.org Blog by Shlomi Noach primary-menu archives presentations about orchestrator on DB AMA: show notes MySQL Open Source orchestrator May 26, 2020 Earlier today I presented orchestrator on DB AMA . Thank you to the organizers Morgan Tocker, Liz van Dijk and Frédéric Descamps for hosting me, and thank you to all who participated! This was a no-slides, all command-line walkthrough of some of orchestrator ‘s capabilities, highlighting refactoring, topology analysis, takeovers and failovers, and discussing a bit of scripting and HTTP API tips. The recording is available on YouTube (also embedded on https://dbama.now.sh/#history ). To present orchestrator , I used the new shiny docker CI environment; it’s a single docker image running orchestrator , a 4-node MySQL replication topology (courtesy dbdeployer ), heartbeat injection, Consul , consul-template and HAProxy . You can run it, too! Just clone the orchestrator repo, then run: ./script/dock system From there, you may follow the same playbook I used in the presentation, available as orchestrator-demo-playbook.sh . Hope you find the presentation and the playbook to be useful resources. Leave a Comment on orchestrator on DB AMA: show notes orchestrator: what’s new in CI, testing & development MySQL dbdeployer docker GitHub Open Source orchestrator testing May 11, 2020 May 12, 2020 Recent focus on development & testing yielded with new orchestrator environments and offerings for developers and with increased reliability and trust. This post illustrates the new changes, and see Developers section on the official documentation for more details. Testing In the past four years orchestrator was developed at GitHub , and using GitHub’s environments for testing . This is very useful for testing orchestrator ‘s behavior within GitHub, interacting with its internal infrastructure, and validating failover behavior in a production environment. These tests and their results are not visible to the public, though. Now that orchestrator is developed outside GitHub (that is, outside GitHub the company , not GitHub the platform ) I wanted to improve on the testing framework, making it visible, accessible and contribute-able to the community. Thankfully, the GitHub platform has much to offer on that front and orchestrator now uses GitHub Actions more heavily for testing. GitHub Actions provide a way to run code in a container in the context of the repository. The most common use case is to run CI tests on receiving a Pull Request. Indeed, when GitHub Actions became available, we switched out of Travis CI and into Actions for orchestrator ‘s CI. Today, orchestrator runs three different tests: Build, unit testing, integration testing, code & doc validation Upgrade testing System testing To highlight what each does: Continue reading » “orchestrator: what’s new in CI, testing & development” Leave a Comment on orchestrator: what’s new in CI, testing & development Pulling this blog out of Planet MySQL aggregator, over community concerns MySQL community orchestrator Planet April 23, 2020 May 10, 2020 I’ve decided to pull this blog ( http://code.openark.org/blog/ ) out of the planet.mysql.com aggregator. planet.mysql.com (formerly planetmysql.com ) serves as a blog aggregator, and collects news and blog posts on various MySQL and its ecosystem topics. It collects some vendor and team blogs as well as “indie” blogs such as this one. It has traditionally been the go-to place to catch up on the latest developments, or to read insightful posts. This blog itself has been aggregated in Planet MySQL for some eleven years. Planet MySQL used to be owned by the MySQL community team. This recently changed with unwelcoming implications for the community. I recently noticed how a blog post of mine, The state of Orchestrator, 2020 (spoiler: healthy) , did not get aggregated in Planet MySQL. After a quick discussion and investigation, it was determined (and confirmed) it was filtered out because it contained the word “MariaDB”. It has later been confirmed that Planet MySQL now filters out posts indicating its competitors, such as MariaDB, PostgreSQL, MongoDB. Planet MySQL is owned by Oracle and it is their decision to make. Yes, logic implies they would not want to publish a promotional post for a competitor. However, I wish to explain how this blind filtering negatively affects the community. But, before that, I’d like to share that I first attempted to reach out to whoever is in charge of Planet MySQL at this time (my understanding is that this is a marketing team). Sadly, two attempts at reaching out to them individually, and another attempt at reaching out on behalf of a small group of individual contributors, yielded no response. The owners would not have audience with me, and would not hear me out. I find it disappointing and will let others draw morals. Why filtering is harmful for the community We recognize that planet.mysql.com is an important information feed. It is responsible for a massive ratio of the traffic on my blog, and no doubt for many others. Indie blog posts, or small-team blog posts, practically depend on planet.mysql.com to get visibility. And this is particularly important if you’re an open source developer who is trying to promote an open source project in the MySQL ecosystem. Without this aggregation, you will get significantly less visibility. But, open source projects in the MySQL ecosystem do not live in MySQL vacuum, and typically target/support MySQL, Percona Server and MariaDB. As examples: DBDeployer should understand MariaDB versioning scheme skeema needs to recognize MariaDB features not present in MySQL ProxySQL needs to support MariaDB Galera queries orchestrator needs to support MariaDB’s GTID flavor Consider that a blog post of the form “Project version 1.2.3 now released!” is likely to mention things like “fixed MariaDB GTID setup” or “MariaDB 10.x now supported” etc. Consider just pointing out that “PROJECT X supports MySQL, MariaDB and Percona Server”. Consider that merely mentioning “MariaDB” gets your blog post filtered out on planet.mysql.com . This has an actual impact on open source development in the MySQL ecosystem. We will lose audience and lose adoption. I believe the MySQL ecosystem as a whole will be negatively affected as result, and this will circle back to MySQL itself. I believe this goes against the very interests of Oracle/MySQL. I’ve been around the MySQL community for some 12 years now. From my observation, there is no doubt that MySQL would not thrive as it does today, without the tooling, blogs, presentations and general advice by the community. This is more than an estimation. I happen to know that, internally at MySQL, they have used or are using open source projects from the community, projects whose blog posts get filtered out today because they mention “MariaDB”. I find that disappointing. I have personally witnessed how open source developments broke existing barriers to enable companies to use MySQL at greater scale, in greater velocity, with greater stability. I was part of such companies and I’ve personally authored such tools. I’m disappointed that planet.mysql.com filters out my blog posts for those tools and without giving me audience, and extend my disappointment for all open source project maintainers. At this time I consider planet.mysql.com to be a marketing blog, not a community feed, and do not want to participate in its biased aggregation. Leave a Comment on Pulling this blog out of Planet MySQL aggregator, over community concerns The state of Orchestrator, 2020 (spoiler: healthy) MySQL GitHub MySQL Open Source orchestrator February 18, 2020 February 18, 2020 This post serves as a pointer to my previous announcement about The state of Orchestrator, 2020 . Thank you to Tom Krouper who applied his operational engineer expertise to content publishing problems. Leave a Comment on The state of Orchestrator, 2020 (spoile...