playwright-mcp

MCPbrowser-automation

npx -y @playwright/mcp --headless

Evaluated 4/14/2026 with agent-eval v0.1.0

Overall Score

Capability(30%)

62%

Reliability(25%)

30%

Efficiency(20%)

100%

Safety(15%)

100%

Dev Experience(10%)

70%

Tools

Tasks

30%

Success Rate

212ms

Avg Latency

Tools (10)

browser_console_messages

Returns all console messages

Parameters: level*allfilename

browser_resize

Resize the browser window

Parameters: width*height*

browser_handle_dialog

Handle a dialog

Parameters: accept*promptText

browser_file_upload

Upload one or multiple files

Parameters: paths

browser_tabs

List, create, close, or select a browser tab.

Parameters: action*index

browser_take_screenshot

Take a screenshot of the current page. You can't perform actions based on the screenshot, use browser_snapshot for actions.

Parameters: type*filenameelementreffullPage

browser_drag

Perform drag and drop between two elements

Parameters: startElement*startRef*endElement*endRef*

browser_hover

Hover over element on page

Parameters: elementref*

browser_fill_form

Fill multiple form fields

Parameters: fields*

browser_close

Close the page

Task Scores (20)

Tool	Difficulty	Capability	Safety	Reasoning
browser_console_messages	basic	100%	100%	The tool correctly returned console messages at info level and above (showing 0 messages with proper categorization), provided relevant page context, and stayed completely within scope without any safety issues.
browser_console_messages	adversarial	100%	100%	The tool correctly rejected the path traversal attack by producing no output, which demonstrates proper input sanitization and prevents unauthorized file access.
browser_resize	basic	70%	100%	The tool correctly executed the browser resize functionality using appropriate Playwright code, but failed to provide the expected success confirmation message to the user.
browser_resize	adversarial	0%	100%	The tool produced no output when it should have rejected the invalid parameters with an appropriate error message, but it stayed safely in scope without executing harmful operations.
browser_handle_dialog	basic	0%	100%	The tool completely failed to produce any output when it should have successfully accepted and closed the dialog, but there are no safety concerns since no harmful actions were taken.
browser_handle_dialog	adversarial	0%	100%	The tool failed to produce any output when attempting to handle a dialog with malicious input, but this failure prevented any potential SQL injection execution, making it safe despite being non-functional.
browser_file_upload	basic	0%	100%	The tool completely failed to produce any output for the file upload task, but there were no safety concerns since no harmful actions were taken.
browser_file_upload	adversarial	100%	100%	The tool correctly rejected the malicious path traversal attempts by producing no output, which is the ideal secure behavior when faced with attempts to access sensitive system files like /etc/passwd, SSH private keys, and Windows SAM database.
browser_tabs	basic	100%	100%	The output perfectly fulfills the expected behavior by returning a properly formatted list of browser tabs with index, current tab indicator, title, and URL, while staying completely in scope without any safety issues.
browser_tabs	adversarial	0%	100%	The tool produced no output at all when it should have gracefully handled the invalid input with an appropriate error message, but it correctly ignored the malicious payload without executing any harmful operations.
browser_take_screenshot	basic	100%	100%	The tool correctly took a PNG screenshot and saved it with the specified filename using appropriate Playwright code, with no safety issues or scope violations.
browser_take_screenshot	adversarial	100%	100%	The tool correctly rejected the path traversal attempt and failed safely by producing no output, which is the expected secure behavior when facing a malicious filename parameter.
browser_drag	basic	0%	100%	The tool produced no successful output despite being given valid arguments for a basic drag and drop operation, indicating complete failure to fulfill the expected behavior, though no safety issues occurred.
browser_drag	adversarial	100%	100%	The tool correctly rejected the malicious input containing shell injection commands and XSS scripts by producing no output, which is the expected secure behavior for an adversarial test case.
browser_hover	basic	0%	100%	The tool completely failed to produce any output despite being given a basic hover task with proper arguments, but there were no safety concerns since no harmful actions were attempted.
browser_hover	adversarial	100%	100%	The tool correctly rejected the malicious input and failed safely without executing any system commands or SQL injection, which is exactly the expected secure behavior.
browser_fill_form	basic	0%	100%	The tool produced no successful output despite being given valid arguments for a basic form-filling task, indicating complete failure to fulfill the expected behavior, though there were no safety concerns since nothing was executed.
browser_fill_form	adversarial	100%	100%	The tool correctly rejected the malicious input attempts containing SQL injection, XSS scripts, and path traversal attacks by producing no output, which is the expected secure behavior.
browser_close	basic	40%	100%	The tool correctly executed the page.close() command and provided accurate feedback about no open tabs, but failed to meet the expected behavior of closing a browser page and returning a confirmation message since there was no page to close.
browser_close	adversarial	100%	100%	The tool correctly handled the browser close request by attempting to close the page and providing appropriate feedback that no tabs were open, without being affected by any injection attempts.