Automation enables bot operators to launch attacks at scale and quickly pivot to evade detection. In a race to defend web, mobile, and API channels against these evolving bot attacks, several anti-bot technologies have risen to the occasion. However, motivated bot operators have become highly sophisticated and use multiple techniques to mimic human behavior and fly beneath the radar of traditional bot mitigation solutions.

Modern anti-bot solutions leverage creative tactics to accurately identify these advanced techniques and ultimately disrupt bot operators. The first step in disrupting the never-ending game of cat and mouse is to understand the players.

Understanding the Cat 

All serious bot mitigation providers inject JavaScript (JS) into the browser to capture data and send it back to detection logic for decisioning. Why is this so important? Client-side JS inspection exposes a different dimension of data that allows for detecting bots in real-time before a session is established. Without JS in the browser to accurately and quickly collect telemetry, you’ll always be at a severe disadvantage. This is because without client-side detection, you’ll need to build a profile of the request data before you can have confidence in your detection decision – it’s already too late.

Such client-side data collection is designed to detect the trace elements of browser automation frameworks that are indicative of a bot. Successfully passing these tests generates a “human” token that provides access to the protected website.

First-generation anti-bot solutions have been criticized for their use of JavaScript code in client-side detection due to the bot operators’ ability to reverse engineer their detection logic. Once a bot understands the detection logic they can send fake data to bypass the detection and successfully launch automated attacks and commit fraud. By using fake data, the bots are polluting the data sets used to differentiate between bots and humans.

Modern defensible JavaScript inspection is designed to prevent bot operators from reverse engineering the detection logic. There are a range of defence techniques required, including sophisticated obfuscation and dynamic detection logic.

These criticisms largely come from anti-bot vendors who only use server-side detection, arguing that JS client-side detection is the old way of identifying advanced persistent bots. The truth is that there is no silver bullet when it comes to bot mitigation. In order to effectively identify and stop sophisticated bot attacks, accurate real-time defensible client-side detection and server-side detection are both required — and need to work together to disrupt attackers.

How Bot Detection Works

Example data flow of a modern anti-bot provider: (1) Client-side scripts execute inside the browser, (2) The browser sends signal data to the telemetry endpoint of the bot mitigation solution, and (3) Server-side detection logic leverages telemetry and threat intelligence to determine whether the request is from a human or bot.

Most anti-bot providers also include “dynamic” challenges to continually revalidate that the requests are being sent from within a browser. For example, a commonly used dynamic challenge is to monitor mouse clicks, movements, and screen interactions. These signals provide validation of human activity by looking for randomisation and variations in movements.

Example of bot-generated mouse movement vs. human mouse movement.

Understanding the Mouse

Bot operators need to obtain a human token and continually imitate human behaviours while conducting their operations. To accomplish this, they attempt to exploit and bypass the anti-bot code that is executed client-side on public devices.

They have two main options to bypass the challenges that the bot mitigation provider uses to differentiate between humans and bots at scale:

  1. Browser-based bots: use heavily customised versions of the leading automation frameworks, such as Playwright, Puppeteer (Stealth), or Electron. Alternatively, a bot operator may use a fully customised browser. We’ve previously explained why bot builders love these frameworks to fly beneath the radar of anti-bot systems.
  2. Request bots: use non-browser based scripts to generate and maintain valid “human” tokens by replaying the expected human telemetry to an anti-bot provider. With request bots, the bot builder is applying the benefits of automation towards bypassing the anti-bot system itself.

What Are Request Bots? And How Do They Work?

Unlike browser-based bots, request bots bypass bot mitigation solutions without using browsers or automation frameworks. In its most simple form, a request bot is built by replaying telemetry outside a browser to generate tokens that mimic human behavior to evade detection of bot mitigation solutions.

Step 1 – Generate tokens:

Bot operators will first deobfuscate an anti-bot provider’s detection script and reverse engineer the detection logic and application flow. Once the logic and structure of the anti-bot model are understood, the bot operator can craft and replay requests that contain the required data to ensure that a human classification + token is provided, and in doing so also pollutes the anti-bot provider’s data source used for analytical modeling.

By reverse engineering the logic of the anti-bot providers model, the bot operator can avoid using a browser-based bot. The replayed requests can be written and executed in the coding language of choice. There are many benefits in avoiding the need to execute within a browser, including throughput, cost, complexity, and skill.

Step 2 – Pass dynamic validations:

Once the bot operator can view the deobfuscated version of the anti-bot script and view the unencrypted data being sent back to the provider, they can reverse engineer the dynamic validations that are occurring.

When presented with a mouse movement obstacle, a bot operator generates (or buys) fake mouse movements. This can be easily accomplished using a self-generating randomised human activity script, recording legitimate activity, or paying for a subscription. One example of a resource to generate human-like mouse movement is Ghost Cursor.

Bypass Bot Mitigation - Buy Mouse Movement

Example of a mouse movement generator that only requires 20 lines of code from the sneakerdev crew.

Once the bot operator has reverse engineered the script and request logic, this fake data can be added to the request bot’s fake replayed requests to defeat the anti-bot provider’s validation process. Check out our infographic to see a visual representation of how request bots work.

Why Can’t My Anti-Bot Provider Detect These Request Bots?

Many bot management providers struggle to detect request bots, as the bot operators supply the anti-bot providers’ system with the seemingly legitimate data that tricks the system to validate the request as human. But this is only possible because the design of the anti-bot system allows for the data to be reliably faked and replayed. So there’s a reasonable chance that your anti-bot provider isn’t detecting these request bots, and unfortunately, you wouldn’t know any better.

We see two primary reasons bot mitigation providers fail to accurately detect request bots:

  1. Failure to safeguard the detection logic that resides within client-side JavaScript with adequate obfuscation. We’ve covered the lack of obfuscation sophistication present in most first-generation bot detection solutions and its impact on the efficacy of those solutions.
  2. Use of static detection logic that allows bot operators to hard-code a request bot and replay data. This model is perfect for request bots to succeed, as the bot operator can predict exactly what is needed and hard-code or statically script their requests accordingly. Most providers will provide the same structure and detection logic in every script for every customer.

The best way to know whether your provider can detect request bots is by searching GitHub and joining Discord groups for documented bypasses. If a bypass exists, this is suggestive that the JavaScript has been deobfuscated. You could also ask your anti-bot provider about their susceptibility to replay attacks – particularly whether their token generation is based on static or dynamic detection logic.

What Does Kasada Do Differently?

Kasada’s innovative take on bot detection is purpose-built to prevent bot operators from successfully using request bots to evade detection, in addition to detecting stealthy browser-based bots. Our platform uses both client-side detection and server-side data analysis, which share accurate information and secure data with one another.

There are multiple layers to our model that disrupt the bot operators’ modus operandi:

  1. Strong defense against reverse engineering – We start by making our detection logic and telemetry data extremely difficult to reverse engineer. Executing detection logic inside a JavaScript virtual machine ensures that the detection model is protected and that the telemetry data is encrypted. This prevents bot operators from generating and replaying the requests.
  2. Dynamic and flexible architecture with reliable data – We further our protection from replay requests with dynamic detection logic. This allows us to present a moving target defence – the detection logic can change at any point, which is a highly efficient way to break a request bot.
  3. Superior protection against advanced automation – By randomising the structure of our scripts, Kasada ensures that bots cannot automate against our scripts. This removes the possibility of building a request bot as each request needs to honour the structure of the individual instance of our script.

Kasada’s modern platform has benefited by learning from the first generation of anti-bot systems that are architected in a way that makes them susceptible to bypass using request bot methods and incapable of quickly making changes. We’ve overcome such architectural flaws by developing new means of protecting client-side JavaScript and incorporating dynamic elements into our detection scripts to ensure that request bots are ineffective. This is one of the many reasons Kasada is able to stop the advanced persistent bots that others can’t, with more than 85% of our customers having tried another bot mitigation platform prior to contacting us.

Don’t just take our word for it – see what our customers have to say.

“When evaluating technology providers, we selected Kasada’s solution for its innovative architecture and its immersive 24/7 customer service that is best characterized as ‘embedded’. We find Kasada to be one of our most valuable controls within our ecosystem.”  – Benjamin Vaughn, VP & CISO, Hyatt Hotels Corp

Kasada’s bot defense platform helps us prepare for whatever the latest tactics and attacks are, stop them before they can cause any damage, and deter future attempts to infiltrate our operations. Put simply, Kasada works.” – Howard Blumenthal, Sr. Director eCommerce, Kaman Distribution Group

From the moment we switched on the platform, there was immediate feedback on the number of page requests that were bot-driven, and I can tell you Kasada neutralized them from the very first page load request. This has enabled us to provide a secure customer experience without the added friction normally found in other solutions.”  – Phil Hawkins, Chief Operating Officer, Flybuys

Join our upcoming Stealth Mode Virtual Meetup on this topic, where we will show you how bot mitigation solutions are being worked around by request bots. Register here!

Want to see Kasada in action? Request a demo here. You can also run an instant test to see if your website can detect modern bots, including those leveraging open source Puppeteer Stealth and Playwright automation frameworks.