prompt-review  by tokoroten

AI agent conversation analysis for understanding user intent and skill

Created 1 week ago

New!

332 stars

Top 82.6% on SourcePulse

GitHubView on GitHub
Project Summary

<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> This tool addresses the challenge of understanding user intent and technical comprehension when generative AI is used in development workflows. It analyzes AI conversation histories to infer technical understanding, prompting patterns, and AI dependency, generating detailed Japanese reports for managers and researchers. The benefit lies in providing objective insights into a user's cognitive state and AI interaction style, moving beyond just code output.

How It Works

<2-4 sentences on core approach / design (key algorithms, models, data flow, or architectural choices) and why this approach is advantageous or novel.> The core approach assumes that user intent and understanding are embedded within their prompts to AI agents. prompt-review automates the collection and analysis of interaction logs from various AI coding assistants. By examining prompt content, it infers the user's technical knowledge level (e.g., mastery, basic understanding, learning), identifies prompting strengths and weaknesses, and categorizes their AI usage style (proactive vs. dependent). This method offers a novel way to assess developer skills and learning trajectories in AI-augmented environments.

Quick Start & Requirements

  • Primary install / run command (pip, Docker, binary, etc.). Execute the /prompt-review command within the Claude Code environment (CLI or VS Code extension).
  • Non-default prerequisites and dependencies (GPU, CUDA >= 12, Python 3.12, large dataset, API keys, OS, hardware, etc.). Claude Code (CLI or VS Code extension), Python 3.10+, SQLite3 (required for parsing GitHub Copilot Chat logs).
  • Estimated setup time or resource footprint. Requires integration with Claude Code and access to AI interaction logs. Specific setup time is not detailed but depends on log availability.
  • If they are present, include links to official quick-start, docs, demo, or other relevant pages. Sample reports are available via a Gist link (not provided in snippet).

Highlighted Details

  • Generates comprehensive Japanese reports detailing data source summaries, project-specific overviews, and technical understanding maps.
  • Evaluates prompting skills, highlighting strengths, areas for improvement, and characteristic user habits.
  • Categorizes AI usage styles, distinguishing between proactive utilization, dependency, and tool-specific tendencies.
  • Tracks the user's growth trajectory over time and offers personalized learning suggestions.

Maintenance & Community

No specific details regarding contributors, sponsorships, community channels (like Discord/Slack), or roadmap were present in the provided README snippet.

Licensing & Compatibility

The license type and compatibility notes for commercial use or closed-source linking were not specified in the provided README snippet.

Limitations & Caveats

The tool is primarily designed to function as a Claude Code skill, suggesting tight integration. Analysis relies on the availability and format of AI interaction logs, which may vary or be incomplete across supported tools. The generated reports are in Japanese, potentially limiting accessibility for non-Japanese speakers.

Health Check
Last Commit

1 week ago

Responsiveness

Inactive

Pull Requests (30d)
10
Issues (30d)
1
Star History
338 stars in the last 12 days

Explore Similar Projects

Feedback? Help us improve.