Lichess End of Year Update 2021
Happy new year!
2021 has been another big year for Lichess - with continued growth, tons of new features, and a fair bit of structural change as an organisation. Thanks to everyone who made this possible!
New Features
We had just released puzzles v2 at the end of 2020, so now it was time to use the new puzzle set. First with the puzzle dashboard, allowing you to analyse your strengths and weaknesses, then with new solving modes, Puzzle Storm, Puzzle Streak, and finally by introducing competitive solving with Puzzle Racer.
2021 also saw many improvements to analysis tools, with Stockfish NNUE now available for analysis in the browser, later updated to Stockfish 14, annotations for inaccuracies/mistakes/blunders on the board, Wikibooks integration, and the opening explorer (recently restored after a data center fire) now with an updated master game database and filterable by player.
We upgraded broadcasts, so that you can more easily follow events brought to you by the new broadcast team, and introduced community blogs for even more chess content.
Also check out the changelog (updated just now) for many more features and bug fixes, including a new setting to hide all ratings on the site so that you can enjoy chess without worrying about numbers.
Content and Events
It’s been a busy year for chess, and as in previous years we’ve hosted our own events and covered some of the biggest events of the year - and Lichess has continued to grow, with several records smashed. We're all incredibly proud, even with the growth sometimes causing some problems.
The Lichess Titled Arenas have continued fortnightly over the past year, with a total of $26,000 paid out in prize money, directly supporting chess players and the chess world in general, and our streamer battles continue their popularity and competitiveness! The twice weekly Lichess Bundesliga regularly has around 500 teams and 5,000 players competing together in 49 leagues. In 192 editions since March 2020, there have been a total of almost a million participations. Our "Lichess Plays" streams where you can play top players while they stream on our Twitch channel have also been popular, with continuing growth, we're glad you're enjoying them! The community run variant world championships also continued with style, thank you to the organisers! There's been a great turnout of very high level players and commentators - offering a great introduction to the variant chess world for viewers.
In addition to covering our own events, Lichess also covered various events in the chess calendar in 2021. Most notable, perhaps, was the World Championship 2021 played between Magnus Carlsen and Ian Nepomniachtchi. Lichess recapped each game, with annotations of the entire match provided by GM Nijat Abasov and made into a study.
During the match, what seems to be the most accurate game ever played in World Championship history was played - with the article and findings Lichess came to, being picked up by renowned statistician site, fivethirtyeight.
Lichess also covered each round of the Chess World Cup, with on-site commentary and streams provided by IM Laura Unuk, who also interviewed players from many of the rounds.
Away from chess events, we're proud to have had the team behind Maia, a human-like neural network chess engine trained on games played on Lichess, share their work and research. Maia is currently the engine that plays in the most human-like way, with 53% of its moves matching first choice human moves for the rating range selected.
We were happy to support Stockfish and Leela Chess Zero (Lc0), by covering the abuse of their free/libre open-source licenses by ChessBase. Lichess is always glad to support the open-source community, as much as it can. Even when it involves the less fun activity of drawing attention to companies disrespecting the legal rights of open source creators!
Lichess was also the first to critically cover the sponsorship deal arranged between FIDE (and specifically women’s chess) and a breast implant manufacturer. The sponsorship provoked widespread controversy not just in chess, but also in mainstream media around the world, with many citing and referencing our work - a testament to the high standard of our content. Lichess was glad to have played its part in attempting to ensure good governance in chess organisations, and not bring the sport into ridicule; virtually no mainstream media was supportive of the sponsorship deal.
And finally, whilst not specifically content in its usual form, Lichess recently instituted a massive rewrite of its Terms of Service (TOS), effective 28th December 2021. Multiple sections were updated, with new unofficial policies and rules which had evolved over time being centrally codified.
Organisation
Growing as an organisation may be even more challenging than writing scalable software. Lichess now has an expenditure of around $33,000 per month to keep everything running (a cost of $0.0002 per game). Bearing in mind that Lichess started as one French developer's hobby project, it humbles the team how far we continue to go, fully funded solely by user donations, with no adverts, trackers, and being free and open source for everyone. We couldn't do this without our community - thank you so much for your ongoing support and encouragement!
A huge amount of work behind the scenes goes into keeping Lichess running smoothly. Thanks to the support we've had, we've been able to hire more people than just the single developer we've historically had, to ensure the work that needs to be done to maintain and grow Lichess can be done. These have been in all areas; developers to maintain the code, fix bugs, and create new features; moderators to keep the site clean from cheaters and those who breach our TOS; content and community, to support our user base, streamers, teachers, coaches, and create written and visual content for us to share with you; and administration - necessary evils like finance, legal, and our internal coordination and operation.
As part of this, we also moved our internal communications from Slack and Discord to Zulip (highly recommended open source tool for distributed teams!), joining our entire team on a single self-hosted communication platform, and supporting fellow projects in the free software community in the process! Nevertheless, always keeping up with everything that happens is nearly impossible at our current size, so we're currently exploring and enacting new ways of working to ensure our community and volunteers are empowered to make the decisions they have the specialised knowledge to make.
Moving on from internals, we've also progressed on working with other organizations. We received an endorsement from the US Chess Federation for our fair play methodology, allowing us to host US Chess events.
Outreach included closely working with schools and educational organizations, some of the work we've done there is:
- Evaluated and updated our data protection policies to ensure ongoing compliance with GDPR - we're currently scoping out COPPA to see how much is already covered;
- In cooperation with Hautes Études des Technologies de l'Information et de la Communication (HETIC), Lichess gave guidance to a group of MA Design students for a week, before helping grade their final submissions - which were all Lichess themed!
- Several pre-schools, schools, colleges, and libraries across the US have accepted our data protection compliance to be sufficient or beyond their needs - and use Lichess, entirely for free, to teach their school kids, students and communities how to play chess;
- Being GDPR compliant and based in France, EU and UK (for now…) based schools, colleges, and universities continue to use Lichess, or have signed up for Lichess in 2021, to teach their school kids, students and community groups how to play chess, entirely for free;
- We have attended various conferences, seminars, and discussions to determine how to better support kids learning chess, the tools coaches and teachers may need, and wider access to chess;
- Ongoing support for U18 tournaments (as well as to other tournaments, too) - both new and old.
Additionally, we've begun being supported by some household names who want to help us do what we do:
- In recognition of Lichess as a public interest web property, Cloudflare is now graciously sponsoring delivery of static files as part of Project Galileo - many thanks to the Cloudflare team for their advocacy and support of the work we do;
- As a recognised French charity, AWS has kindly provided us with free credit which meets some of our computing needs;
- Our long-term hosting provider, OVH (based in France!) also kindly recognised the work we do as being in the public interest, providing us with excellent support and enabling us to reduce our server costs;
- Huawei also continued their history of supporting free software / open-source projects, with Lichess.org being selected as a Speed Dial partner website from June - December 2021.
Our software, as in previous years, supported everything from hobby projects to major commercial releases. We love hearing about how our code has been adapted and used, and how it supports new projects - feel free to tell us about it, or share how your existing project has grown!
Technical Updates
After struggling to scale up our infrastructure with the growth in 2020, we were well prepared for most of 2021, despite new record numbers in games played, early in the year.
The first emergencies of 2021 were MongoDB crashing in production once, and later again after dropping support for deeply nested objects. We were using these to store the tree of variations in studies, so that we had to rush a rewrite to a flattened storage format. Before these incidents, and since then, MongoDB has been reliable as usual. In the course of the rewrite, we also moved studies into a separate independent database.
Next, there was some time to consolidate. Fixing small bugs, adding type annotations to remaining untyped JavaScript code, switching from CommonJS to ESM for moderate bundle size savings, replacing the PHP-based OAuth server with a more robust Scala implementation (now allowing public clients via PKCE), and moving expendable collections out of the main database to an unreplicated MongoDB database, so that secondaries of the main database can keep up more easily with the oplog.
Now, towards the end of the year, we're again hitting new all-time highs, not in number of games played per month, but in numbers of spectators watching the World Championship broadcast and the World Rapid and Blitz broadcasts, as well as concurrent players and spectators in Agadmator's CakeDeFi Christmas Arena, posing new challenges. We started by optimizing and carefully parallelizing the previously sequential processing of tournament events. Better scaling with the numbers of spectators will be the first technical goal of 2022.
Epilogue
We hope you enjoyed Lichess in 2021 and wish you a happy new year!