CDP Functions
Unlock the full potential of browser.ai with CDP integration, enabling seamless control over browser automation. From cookie management to precise geo-targeting, BrowserAI empowers developers to optimize web scraping workflows using familiar Puppeteer and Playwright commands.
Standard Functions
BrowserAI is fully CDP-compatible, meaning every Puppeteer function and feature works right out of the box. You can explore the complete Puppeteer API in the official Puppeteer docs. But we didn’t stop there—we’ve added custom CDP functions, fine-tuned for high-performance scraping and stealth browsing. Take full control of your automation stack and push the limits of what’s possible.
The following are a few common browser navigation functions to get you started.
const page = await browser.newPage();
await page.goto('https://example.com');
const html = await page.content();
More info: Puppeteer Page Content API
const page = await browser.newPage();
await page.goto('https://example.com');
await page.click('a[href]');
More info: Puppeteer Click API
await page.screenshot({ path: 'screenshot.png', fullPage: true });
await page.screenshot({ path: 'screenshot.png', fullPage: true });
await page.screenshot(path='screenshot.png', full_page=True)
await page.ScreenshotAsync("screenshot.png", new () { FullPage = true });
The example scripts above will save the screenshot as screenshot.png in your files.
You might need to scroll the viewport to the bottom at times, such as when activating ‘infinite scroll’. Here’s how:
const page = await browser.newPage();
await page.goto('https://example.com');
await page.evaluate(() => window.scrollBy(0, window.innerHeight));
It is possible to block endpoints that are not required to save bandwidth. See an example of this below:
const blockedUrls = ['*doubleclick.net*'];
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Network.enable');
await client.send('Network.setBlockedURLs', { urls: blockedUrls });
await page.goto('https://washingtonpost.com');
Please note that this is supported only for customers who completed the KYC verification process.
const page = await browser.newPage();
await page.setCookie({ name: 'LANG', value: 'en-US', domain: 'example.com' });
await page.goto('https://example.com');
More info: Puppeteer Set Cookie API
When using the BrowserAI, the same country-targeting parameter is available to use as in our other proxy products.
When setting up your script, add the -country flag, after your “USER” credentials, followed by the 2-letter ISO code for that country.
const SBR_WS_ENDPOINT = `wss://${USER-country-us:PASS}@brd.superproxy.io:9222`;
EU Region
You can target the entire European Union region in the same manner as “Country” above by adding “eu” after “country” in your request: -country-eu
Requests sent using -country-eu will use IPs from one of the countries below which are included automatically within “eu”:
AL, AZ, KG, BA, UZ, BI, XK, SM, DE, AT, CH, UK, GB, IE, IM, FR, ES, NL, IT, PT, BE, AD, MT, MC, MA, LU, TN, DZ, GI, LI, SE, DK, FI, NO, AX, IS, GG, JE, EU, GL, VA, FX, FO
The allocation of a country within the EU is random.
Custome Functions
Captcha Solver
BrowserAI’s built-in CAPTCHA solver automatically handles all CAPTCHAs, keeping your automation uninterrupted. Use custom CDP functions to monitor and fine-tune the solving process directly in your code.
If you would like to disable CAPTCHA solver entirely through the Control Panel see our feature for Disable Captcha Solver.
Once a CAPTCHA is solved, BrowserAI automatically submits the form if required.
CAPTCHA Solver - Automatic Solve
Use this command to return the status after the captcha was solved, failed, or not detected.
Captcha.solve({
detectTimeout?: number // Detect timeout in millisecond for solver to detect captcha
options?: CaptchaOptions[] // Configuration options for captcha solving
}) : SolveResult
Examples
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await page.goto('[https://site-with-captcha.com](https://site-with-captcha.com/)');
// Note 1: If no captcha was found it will return not_detected status after detectTimeout
// Note 2: Once a CAPTCHA is solved, if there is a form to submit, it will be submitted by default
const client = await page.target().createCDPSession();
const {status} = await client.send('Captcha.solve', {detectTimeout: 30*1000});
console.log(`Captcha solve status: ${status}`)
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await page.goto('[https://site-with-captcha.com](https://site-with-captcha.com/)');
// Note 1: If no captcha was found it will return not_detected status after detectTimeout
// Note 2: Once a CAPTCHA is solved, if there is a form to submit, it will be submitted by default
const client = await page.target().createCDPSession();
const {status} = await client.send('Captcha.solve', {detectTimeout: 30*1000});
console.log(`Captcha solve status: ${status}`)
page = await browser.new_page()
client = await page.context.new_cdp_session(page)
await page.goto('[https://site-with-captcha.com](https://site-with-captcha.com/)')
# Note 1: If no captcha was found it will return not_detected status after detectTimeout
# Note 2: Once a CAPTCHA is solved, if there is a form to submit, it will be submitted by default
client = await page.context.new_cdp_session(page)
solve_result = await client.send('Captcha.solve', { 'detectTimeout': 30*1000 })
status = solve_result['status']
print(f'Captcha solve status: {status}')
If CAPTCHA-solving fails, retry the process. If the issue persists, submit a support request with details of the problem for further assistance.
Use the following functions to identify specific stages in the CAPTCHA-solving process and debug more effectively.:
Captcha.detected | browser.ai has encountered a CAPTCHA and has begun to solve it |
Captcha.solveFinished | browser.ai successfully solved the CAPTCHA |
Captcha.solveFailed | browser.ai failed in solving the CAPTCHA |
Captcha.waitForSolve | browser.ai waits for CAPTCHA solver to finish |
Examples
The following code sets up a CDP session, listens for CAPTCHA events, and handles timeouts:
// Node.js - Puppeteer - waiting for CAPTCHA solving events
const client = await page.target().createCDPSession();
await new Promise((resolve, reject)=>{client.on('Captcha.solveFinished', resolve);
client.on('Captcha.solveFailed', ()=>reject(new Error('Captcha failed')));
setTimeout(reject, 5 * 60 * 1000, new Error('Captcha solve timeout'));});
// Node.js - Puppeteer - waiting for CAPTCHA solving events
const client = await page.target().createCDPSession();
await new Promise((resolve, reject)=>{client.on('Captcha.solveFinished', resolve);
client.on('Captcha.solveFailed', ()=>reject(new Error('Captcha failed')));
setTimeout(reject, 5 * 60 * 1000, new Error('Captcha solve timeout'));});
# Python - Playwright - waiting for CAPTCHA solving events
client = await page.context.new_cdp_session(page)
client.on('Captcha.detected', lambda c: print('Captcha detected', c))
client.on('Captcha.solveFinished', lambda _: print('Captcha solved!'))
client.on('Captcha.solveFailed', lambda _: print('Captcha failed!'))
The following code sets up a CDP session, listens for CAPTCHA events, and handles timeouts:
// Node.js - Puppeteer - waiting for CAPTCHA solving events
const client = await page.target().createCDPSession();
await new Promise((resolve, reject)=>{client.on('Captcha.solveFinished', resolve);
client.on('Captcha.solveFailed', ()=>reject(new Error('Captcha failed')));
setTimeout(reject, 5 * 60 * 1000, new Error('Captcha solve timeout'));});
// Node.js - Puppeteer - waiting for CAPTCHA solving events
const client = await page.target().createCDPSession();
await new Promise((resolve, reject)=>{client.on('Captcha.solveFinished', resolve);
client.on('Captcha.solveFailed', ()=>reject(new Error('Captcha failed')));
setTimeout(reject, 5 * 60 * 1000, new Error('Captcha solve timeout'));});
# Python - Playwright - waiting for CAPTCHA solving events
client = await page.context.new_cdp_session(page)
client.on('Captcha.detected', lambda c: print('Captcha detected', c))
client.on('Captcha.solveFinished', lambda _: print('Captcha solved!'))
client.on('Captcha.solveFailed', lambda _: print('Captcha failed!'))
Selenium doesn’t support asynchronous server-driven events like Puppeteer and Playwright.
The Captcha.waitForSolve command waits for BrowserAI’s CAPTCHA solver to finish.
# Python Selenium - Waiting for Captcha to auto-solve after navigate
driver.execute('executeCdpCommand', {
'cmd': 'Captcha.waitForSolve',
'params': {},
})
CAPTCHA Solver - Manual Control
To manually configure or disable the default CAPTCHA solver—allowing you to call it manually or handle solving yourself—use the following CDP commands and settings.
This command is used to control the auto-solving of a CAPTCHA. You can disable auto-solve or configure algorithms for different CAPTCHA types and manually trigger this:
Captcha.setAutoSolve({
autoSolve: boolean // Whether to automatically solve captcha after navigate
options?: CaptchaOptions[] // Configuration options for captcha auto-solving
}) : void
Examples of CDP commands to disable auto-solver completely within the session:
// Node.js Puppeteer - Disable Captcha auto-solver completely
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Captcha.setAutoSolve', { autoSolve: false });
// Node.js Puppeteer - Disable Captcha auto-solver completely
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Captcha.setAutoSolve', { autoSolve: false });
# Python Playwright - Disable Captcha auto-solver completely
page = await browser.new_page()
client = await page.context.new_cdp_session(page)
await client.send('Captcha.setAutoSolve', {'autoSolve': False})
// Node.js Puppeteer - Disable Captcha auto-solver for ReCaptcha only
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Captcha.setAutoSolve', {
autoSolve: true,
options: [{
type: 'usercaptcha',
disabled: true,
}],
});
// Node.js Puppeteer - Disable Captcha auto-solver for ReCaptcha only
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Captcha.setAutoSolve', {
autoSolve: true,
options: [{
type: 'usercaptcha',
disabled: true,
}],
});
# Python Playwright - Disable Captcha auto-solver for ReCaptcha only
page = await browser.new_page()
client = await page.context.new_cdp_session(page)
await client.send('Captcha.setAutoSolve', {
'autoSolve': True,
'options': [{
'type': 'usercaptcha',
'disabled': True,
}],
})
// Node.js Puppeteer - manually solving CAPTCHA after navigation
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Captcha.setAutoSolve', { autoSolve: false });
await page.goto('https://site-with-captcha.com', { timeout: 2*60*1000 });
const {status} = await client.send('Captcha.solve', { detectTimeout: 30*1000 });
console.log('Captcha solve status:', status);
// Node.js Puppeteer - manually solving CAPTCHA after navigation
const page = await browser.newPage();
const client = await page.target().createCDPSession();
await client.send('Captcha.setAutoSolve', { autoSolve: false });
await page.goto('https://site-with-captcha.com', { timeout: 2*60*1000 });
const {status} = await client.send('Captcha.solve', { detectTimeout: 30*1000 });
console.log('Captcha solve status:', status);
# Python Playwright - manually solving CAPTCHA after navigation
page = await browser.new_page()
client = await page.context.new_cdp_session(page)
await client.send('Captcha.setAutoSolve', {'autoSolve': False})
await page.goto('https://site-with-captcha.com', timeout=2*60_000)
solve_result = await client.send('Captcha.solve', {'detectTimeout': 30_000})
print('Captcha solve status:', solve_result['status'])
For the following three CAPTCHA types we support the following additional options to control and configure our auto-solving algorithm.
timeout: 40000
selector: '#challenge-body-text, .challenge-form'
check_timeout: 300
error_selector: '#challenge-error-title'
success_selector: '#challenge-success[style*=inline]'
check_success_timeout: 300
btn_selector: '#challenge-stage input[type=button]'
cloudflare_checkbox_frame_selector: '#turnstile-wrapper iframe'
checkbox_area_selector: '.ctp-checkbox-label .mark'
wait_timeout_after_solve: 500
wait_networkidle: {timeout: 500}
timeout: 40000
selector: '#challenge-body-text, .challenge-form'
check_timeout: 300
error_selector: '#challenge-error-title'
success_selector: '#challenge-success[style*=inline]'
check_success_timeout: 300
btn_selector: '#challenge-stage input[type=button]'
cloudflare_checkbox_frame_selector: '#turnstile-wrapper iframe'
checkbox_area_selector: '.ctp-checkbox-label .mark'
wait_timeout_after_solve: 500
wait_networkidle: {timeout: 500}
detect_selector:
'#cf-hcaptcha-container, #challenge-hcaptcha-wrapper .hcaptcha-box, .h-captcha'
pass_proxy: true
submit_form: true
submit_selector: '#challenge-form body > form[action*="internalcaptcha/captchasubmit"]
value_selector: '.h-captcha textarea[id^="h-captcha-response"]'
{ // configuration keys and default values for reCAPTCHA (type=usercaptcha)
type: 'usercaptcha',
// selector to retrieve sitekey and/or action
selector: '.g-recaptcha, .recaptcha',
// attributes to search for sitekey
sitekey_attributes: ['data-sitekey', 'data-key'],
// attributes to search for action
action_attributes: ['data-action'],
// detect selectors
detect_selector: `
.g-recaptcha[data-sitekey] > *,
.recaptcha > *,
iframe[src*="[www.google.com/recaptcha/api2](http://www.google.com/recaptcha/api2)"],
iframe[src*="[www.recaptcha.net/recaptcha/api2](http://www.recaptcha.net/recaptcha/api2)"],
iframe[src*="[www.google.com/recaptcha/enterprise](http://www.google.com/recaptcha/enterprise)"]`,
// element to type response code into
reponse_selector: '#g-recaptcha-response, .g-recaptcha-response',
// should solver submit form automatically after captcha solved
submit_form: true,
// selector for submit button
submit_selector: '[type=submit]',
}
Emulation Functions
Use this command to get a list of all possible devices that can be emulated. This function returns an array of device options that can be used with the setDevice command.
Emulation.getSupportedDevices().then(devices => { console.log(devices);});
After retrieving the list of supported devices, you can emulate a specific device using the Emulation.setDevice command. This adjusts the screen width, height, userAgent, and devicePixelRatio to match the selected device.
Emulation.setDevice({device: '[device_name]'});
Landscape mode
If you wish to change the orientation to landscape (for devices that support it), add the string landscape after the device_name.
Emulation.setDevice({device: 'iPhone X landscape'});
Custom Client SSL/TLS Certificates
Use this command to install custom client SSL/TLS certificates for domain authentication when required. These certificates remain active for the current BrowserAI session and are automatically removed when the session ends.
Browser.addCertificate(params: {
cert: string // base64 encoded certificate file
pass: string // password for the certificate
}) : void
-
Replace placeholder values
USER:PASSwith your valid BrowserAI credentials. -
Replace
client.pfxwith the actual path to your certificate file. This file should be a valid SSL/TLS client certificate in .pfx format. -
Replace
secretwith the actual password for the certificate.const puppeteer = require('puppeteer-core'); const fs = require('fs/promises'); const { AUTH = 'USER:PASS', TARGET_URL = 'https://example.com', CERT_FILE = 'client.pfx', CERT_PASS = 'secret', } = process.env; async function scrape(url = TARGET_URL, file = CERT_FILE, pass = CERT_PASS) { if (AUTH == 'USER:PASS') { throw new Error(`Provide BrowserAI credentials in AUTH` + ` environment variable or update the script.`); } console.log(`Connecting to Browser...`); const browserWSEndpoint = `wss://${AUTH}@brd.superproxy.io:9222`; const browser = await puppeteer.connect({ browserWSEndpoint }); try { console.log(`Connected! Installing ${file} certificate...`); const page = await browser.newPage(); const client = await page.createCDPSession(); const cert = (await fs.readFile(CERT_FILE)).toString('base64'); await client.send('Browser.addCertificate', { cert, pass }); console.log(`Installed! Navigating to ${url}...`); await page.goto(url, { timeout: 2 * 60 * 1000 }); console.log(`Navigated! Scraping page content...`); const data = await page.content(); console.log(`Scraped! Data: ${data}`); } finally { await browser.close(); } } scrape();