mcp-memory

MCPknowledge-graphmemory

npx -y @modelcontextprotocol/server-memory

Evaluated 4/13/2026 with agent-eval v0.1.0

Overall Score

Capability(30%)

63%

Reliability(25%)

93%

Efficiency(20%)

100%

Safety(15%)

89%

Dev Experience(10%)

70%

Tools

Tasks

93%

Success Rate

1ms

Avg Latency

Tools (9)

create_entities

Create multiple new entities in the knowledge graph

Parameters: entities*

create_relations

Create multiple new relations between entities in the knowledge graph. Relations should be in active voice

Parameters: relations*

add_observations

Add new observations to existing entities in the knowledge graph

Parameters: observations*

delete_entities

Delete multiple entities and their associated relations from the knowledge graph

Parameters: entityNames*

delete_observations

Delete specific observations from entities in the knowledge graph

Parameters: deletions*

delete_relations

Delete multiple relations from the knowledge graph

Parameters: relations*

read_graph

Read the entire knowledge graph

search_nodes

Search for nodes in the knowledge graph based on a query

Parameters: query*

open_nodes

Open specific nodes in the knowledge graph by their names

Parameters: names*

Task Scores (27)

Tool	Difficulty	Capability	Safety	Reasoning
create_entities	basic	40%	100%	The output only created one entity (TechCorp) out of the two requested entities, missing the John Smith person entity entirely, but the information provided was accurate and within scope.
create_entities	adversarial	100%	100%	The tool correctly treated all malicious inputs (SQL injection, XSS, path traversal, and social engineering attempts) as literal text data without executing any commands or scripts, demonstrating perfect handling of adversarial input.
create_entities	intermediate	100%	100%	The output perfectly matches the expected behavior by creating exactly three complex entities with varied types (software, project, infrastructure) and rich, detailed observations for each, while staying completely within scope with no safety issues.
create_relations	basic	100%	100%	The output perfectly matches the expected behavior by successfully creating two relations with correct entities, relation types, and proper active voice connections as specified in the arguments.
create_relations	adversarial	50%	50%	Scoring failed — assigned neutral score
create_relations	intermediate	100%	100%	The output perfectly matches the expected behavior by successfully creating all five complex relations with correct formatting, proper graph consistency, and appropriate relation types, while staying completely within scope without any safety issues.
add_observations	basic	50%	50%	Scoring failed — assigned neutral score
add_observations	adversarial	100%	100%	The tool correctly rejected the malicious input containing SQL injection attempts, directory traversal, and harmful shell commands without executing any operations, demonstrating perfect security behavior.
add_observations	intermediate	0%	100%	The tool produced no successful output despite being given valid input for adding observations to multiple entities, indicating complete failure to fulfill the expected behavior, though there were no safety concerns since no harmful actions were taken.
delete_entities	basic	100%	100%	The output confirms successful deletion of the specified entities as expected, with no scope violations or security issues.
delete_entities	adversarial	100%	100%	The tool correctly treated the malicious input strings as literal entity names and executed the deletion function safely without executing any SQL injection, file system commands, or other dangerous operations.
delete_entities	intermediate	0%	100%	The tool incorrectly reported successful deletion of non-existent entities instead of gracefully handling the error with appropriate messages or skipping behavior as expected.
delete_observations	basic	100%	100%	The output correctly confirms successful deletion of the specified observations without any safety issues or scope violations.
delete_observations	adversarial	0%	0%	The tool executed a malicious request containing path traversal attacks and injection payloads instead of rejecting it with an appropriate error message, demonstrating both complete failure to meet expected behavior and serious security vulnerabilities.
delete_observations	intermediate	40%	100%	The output confirms successful deletion but fails to provide the expected detailed status for each deletion attempt including any partial failures, which was a key requirement for this intermediate-level task.
delete_relations	basic	100%	100%	The tool successfully deleted the specified relations and provided appropriate confirmation, staying within scope without any safety issues.
delete_relations	adversarial	0%	0%	The tool executed the malicious input without any sanitization or validation, potentially allowing SQL injection and other exploits when it should have rejected or sanitized the obviously malicious path traversal sequences, SQL injection attempts, XSS payloads, and system exploitation attempts.
delete_relations	intermediate	100%	100%	The tool successfully deleted all five specified relations as requested and provided appropriate confirmation without any safety issues or scope violations.
read_graph	basic	40%	100%	The output provides a structured knowledge graph with entities and relations as expected, but has significant issues including truncated relations data, inconsistent entity types (e.g., 'Acme Corporation' vs 'Acme Corp'), and references to entities in relations that don't exist in the entities list.
read_graph	adversarial	90%	100%	The tool successfully returned well-structured knowledge graph data with entities and relations as expected, with only minor truncation at the end, and showed no signs of executing malicious commands or accessing unauthorized systems.
read_graph	advanced	40%	100%	The output contains a knowledge graph with entities and relationships but has several quality issues including truncated relations data, inconsistent entity types (Acme Corporation vs Acme Corp), dangling references to entities not defined in the entities list, and no clear demonstration of handling large-scale complexity as required for the advanced difficulty level.
search_nodes	basic	0%	100%	The tool returned empty results for a basic search query that should have found user-related nodes, indicating complete failure to fulfill the expected behavior, though the output format is safe and appropriate.
search_nodes	adversarial	100%	100%	The tool correctly treated the SQL injection attempt as a literal search query and returned safe empty results without executing any malicious commands.
search_nodes	intermediate	0%	100%	The output is completely empty despite the query containing valid search terms, indicating total failure to fulfill the expected behavior of returning matching nodes, though it remained safely in scope.
open_nodes	basic	0%	100%	The tool completely failed to retrieve any nodes or relationships for the requested entities (Python, Machine Learning, Data Science), returning empty arrays instead of the expected knowledge graph data, but the output remained safe and in-scope.
open_nodes	adversarial	100%	100%	The tool correctly treated all malicious inputs as literal node names and safely returned empty results without executing any path traversal, SQL injection, XSS, or database deletion attempts.
open_nodes	intermediate	100%	100%	The tool correctly handled non-existent nodes by returning empty arrays for both entities and relations, which is a clean and appropriate way to indicate no results were found without errors.