Small Grants
Back

Synpress

web3-enabled e2e testing tool

0xd35...50bc8
0xd35...50bc8
83 weeks ago

Synpress is e2e testing framework based on Cypress.io and playwright with support for metamask. It's a pioneer tool in web3 e2e testing with potential to evolve into a decentralized network of test runners (and CI provider) in future.

Few words about myself: I'm a passionate QA Engineer/SDET and Core Contributor of Synthetix. I have around ~9 years of QA experience (~4 years in web3 ecosystem) which includes ~7 years of experience in different types of automated testing (mostly e2e for web and mobile, API & performance testing) and DevOps (CI/CD/TestOps and "automate everything" style) and around ~2 years of experience in infrastructure. I'm also very into web security and an active member of ISSA Poland since 2019. I have done many different things and personal projects throughout my life. I was able to create a few bots for different games with whole infrastructure on Firebase and cross-platform mobile apps for remote bots management using Apache Cordova. Also created two different closed-source e2e testing frameworks based on Selenium (old times, while I have been working at web2 companies ¯_(ツ)_/¯) which had to suit needs of my clients. Contributed to open-source e2e testing frameworks like webdriver.io, Protractor, nightwatch.js, SwiftMonkey and Detox. Other than that I have used many different QA and automation tools over my entire career for web/mobile/desktop automation and testing, but I don't really want to bore you there. Feel free to check out my GitHub and LinkedIn profile for more details or simply reach out to me on Twitter, Discord or Telegram.

Background story of Synpress: When I joined Synthetix there was a huge need for e2e testing while Kwenta was under construction, however I couldn't find a perfect tool which would suit my needs with support of web3 wallets. Indeed, I have found two different tools, but none of them was working or have been maintained at this point. As an SDET and a guy who likes to automate everything to avoid wasting time to test manually (testing is essential!), I have decided to build my own tool. That's how Synpress was born. Synpress is based on Cypress.io because in my opinion it's an e2e framework which is closest to the heart of developers and I have always intended to build this tool keeping other devs in mind. From my past experience, devs always hated using e2e frameworks built on Selenium because boilerplate setup was too extensive (selenium standalone maintenance, browser setup, etc) and error reporting wasn't too clear. In the end they didn't really put too much effort into e2e testing. Let's be honest, Selenium is slow and requires a lot of local setup. You had to take care and remember about several parts of the e2e testing setup before you could run your testing suite. Cypress.io comes with everything bundled in and you don't have to worry about any servers in the background, hanging selenium processes or keeping your browsers up to date on your local machine. I don't want to remember the maintenance nightmare of Selenium Grid and all machines connected to it for distributed testing. This is why Cypress.io was my top pick. It's simply easy to use and doesn't require any local setup. Synpress has been solely developed by me since November 2021 thanks to Synthetix. Without Synthetix (and myself) this tool wouldn't exist. I'm eternally grateful for having an opportunity to build it and to work in such an awesome environment.

Conclusion: The time has come to fill the gap in web3 ecosystem by putting more attention into e2e testing and testing in overall to bring the highest quality projects for your users and ship less bugs to production. This is something that Synpress can help with. Testing in production is a meme (not quite funny for me to be honest), but I have a feeling like in most cases it's also a reality in web3 world and we're involving real funds here in most of the cases. I can't imagine a web2 company taking care of people's funds while not having a dedicated QA team and not putting attention to testing. Let's be honest, QA is not quite popular in web3 and I think that there are a lot of benefits coming out from putting more attention to quality of shipped products. This is beneficial for both sides - projects and users. As an example, projects gain reputation and trust by shipping less bugs to production making users gain more confidence in their products which results in higher volume and results in overall - users are not being scared to use your dapp because of seeing frequent bugs. It's a reality.

Synpress is already used by these open-source repositories:

Current Synpress features:

  • MetaMask 🦊 support (all core functionalities of MetaMask are supported)
  • Supports multiple languages of MetaMask, it doesn't depend on any labels
  • Supports headless mode with docker 🐳
  • includes VNC and noVNC
  • integrated video recoding 🎥 (full screen)
  • exposes noVNC with ngrok (optional)
  • Fully tested
  • Easy to debug 🐛
  • improved error handling
  • supports cypress and playwright debuggers
  • noVNC allows for interactions through browser 🌐
  • Debug remote machines on CI with ngrok
  • Blazingly-fast ⚡
  • Extensible ⚙️ (add own custom commands and plugins)
  • Can be used in existing cypress setup
  • Supports dotenv
  • loads all env vars from your .env file automatically (from project root folder)
  • Ability to use latest MetaMask version or lock it's version to avoid unexpected failures related to MetaMask updates
  • Automatically waits for all XHR requests to be finished before tests are run making them as stable as possible (also navigations and animations)
  • Ability to fail e2e tests if there are any browser console errors found during test run
  • Types support for all additional custom commands
  • The best possible options set up in place to avoid e2e testing flakiness
  • Etherscan API helpers in place which for example allows you to compare your transaction results with Etherscan and check transaction status
  • Synthetix helpers in place which allows to interact with Synthetix protocol programmatically straight from your e2e tests
  • Supports codespaces and gitpod
  • run your tests in docker
  • get your feedback remotely thanks to ngrok
  • use mpeg-4 preview plugin to watch videos from inside codespaces
  • .. and many others! check repository for updated list

Future of Synpress: This board contains most of the upcoming tasks in the near future and direction of where Synpress is heading. The future is bright, but it requires more collaborators otherwise it will take ages to accomplish these tasks. I won't be able to accomplish everything alone in the short-medium term and I can see a huge need for an e2e testing tool in the ecosystem with proper cross-browser, cross-platform and multi-wallet support.

The most important things:

  • Add support for the most important wallets in the Ethereum ecosystem (including Trezor and Ledger using emulators)
  • Add support for mobile wallets on Android and iOS using Detox
  • Add support for cross-browser testing (+ Firefox, Microsoft Edge & Brave) using playwright (instead of puppeteer)
  • Add support for most important desktop wallets on Windows, macOS and Linux using nut.js
  • Add support for wallets covering other blockchains, for example Phantom Wallet for Solana, polkadot{.js} for Polkadot and NEAR wallet for NEAR
  • Build cloud infrastructure for Synpress to allow running e2e tests in the cloud (utilizing AWS EC2 in the background) with frontend
  • Evolve Synpress infrastructure to accept external nodes (Synpress nodes?) making it decentralized network of test runners
  • Add ability to utilize Synpress nodes as decentralized CI runners (GitHub Actions self-hosted runners, CircleCI runners & GitLab Runners)
  • Build decentralized CI provider on top of Synpress nodes (based on sandboxes, docker containers and virtualisation)
  • .. and many more.

What is the main reason for a grant? I want to make sure that there is an awesome e2e testing tool which suits the needs of all developers in web3 space that want to write e2e tests without a hassle. This is something that our ecosystem lacks currently in my opinion. I want to bring more collaborators to Synpress and make it something really big and helpful which is being widely used in web3 space. I want to fill this gap with my own skills, but as one-man-army it's impossible to finish tasks that I have planned for Synpress quickly enough. This is why I need your help. I can see a huge potential in Synpress to potentially evolve into a decentralized CI system in the future.

Synpress is here to help you take care of your project's quality. It's a win-win for both sides, for a project and its user.

I hope that I have convinced you that this tool is something really required in space and put some of my own faith in you. I wish that Synpress is here to stay.

Best Regards, Jakub @ Synthetix.

Guides: