Please enter the commit message for your changes. Lines starting
with '#' will be ignored, and an empty message aborts the commit. On branch main Your branch is up to date with 'origin/main'. Changes to be committed: new file: .claude/skills/algorithmic-art/.openskills.json new file: .claude/skills/algorithmic-art/LICENSE.txt new file: .claude/skills/algorithmic-art/SKILL.md new file: .claude/skills/algorithmic-art/templates/generator_template.js new file: .claude/skills/algorithmic-art/templates/viewer.html new file: .claude/skills/brand-guidelines/.openskills.json new file: .claude/skills/brand-guidelines/LICENSE.txt new file: .claude/skills/brand-guidelines/SKILL.md new file: .claude/skills/canvas-design/.openskills.json new file: .claude/skills/canvas-design/LICENSE.txt new file: .claude/skills/canvas-design/SKILL.md new file: .claude/skills/canvas-design/canvas-fonts/ArsenalSC-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/ArsenalSC-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/BigShoulders-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/BigShoulders-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/BigShoulders-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Boldonse-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Boldonse-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/BricolageGrotesque-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/BricolageGrotesque-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/BricolageGrotesque-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/CrimsonPro-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/CrimsonPro-Italic.ttf new file: .claude/skills/canvas-design/canvas-fonts/CrimsonPro-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/CrimsonPro-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/DMMono-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/DMMono-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/EricaOne-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/EricaOne-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/GeistMono-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/GeistMono-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/GeistMono-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Gloock-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Gloock-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexMono-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexMono-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexMono-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexSerif-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexSerif-BoldItalic.ttf new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexSerif-Italic.ttf new file: .claude/skills/canvas-design/canvas-fonts/IBMPlexSerif-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSans-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSans-BoldItalic.ttf new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSans-Italic.ttf new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSans-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSans-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSerif-Italic.ttf new file: .claude/skills/canvas-design/canvas-fonts/InstrumentSerif-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Italiana-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Italiana-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/JetBrainsMono-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/JetBrainsMono-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/JetBrainsMono-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Jura-Light.ttf new file: .claude/skills/canvas-design/canvas-fonts/Jura-Medium.ttf new file: .claude/skills/canvas-design/canvas-fonts/Jura-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/LibreBaskerville-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/LibreBaskerville-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Lora-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/Lora-BoldItalic.ttf new file: .claude/skills/canvas-design/canvas-fonts/Lora-Italic.ttf new file: .claude/skills/canvas-design/canvas-fonts/Lora-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Lora-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/NationalPark-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/NationalPark-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/NationalPark-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/NothingYouCouldDo-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/NothingYouCouldDo-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Outfit-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/Outfit-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Outfit-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/PixelifySans-Medium.ttf new file: .claude/skills/canvas-design/canvas-fonts/PixelifySans-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/PoiretOne-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/PoiretOne-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/RedHatMono-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/RedHatMono-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/RedHatMono-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/Silkscreen-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Silkscreen-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/SmoochSans-Medium.ttf new file: .claude/skills/canvas-design/canvas-fonts/SmoochSans-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Tektur-Medium.ttf new file: .claude/skills/canvas-design/canvas-fonts/Tektur-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/Tektur-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/WorkSans-Bold.ttf new file: .claude/skills/canvas-design/canvas-fonts/WorkSans-BoldItalic.ttf new file: .claude/skills/canvas-design/canvas-fonts/WorkSans-Italic.ttf new file: .claude/skills/canvas-design/canvas-fonts/WorkSans-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/WorkSans-Regular.ttf new file: .claude/skills/canvas-design/canvas-fonts/YoungSerif-OFL.txt new file: .claude/skills/canvas-design/canvas-fonts/YoungSerif-Regular.ttf new file: .claude/skills/doc-coauthoring/.openskills.json new file: .claude/skills/doc-coauthoring/SKILL.md new file: .claude/skills/docx/.openskills.json new file: .claude/skills/docx/LICENSE.txt new file: .claude/skills/docx/SKILL.md new file: .claude/skills/docx/scripts/__init__.py new file: .claude/skills/docx/scripts/accept_changes.py new file: .claude/skills/docx/scripts/comment.py new file: .claude/skills/docx/scripts/office/helpers/__init__.py new file: .claude/skills/docx/scripts/office/helpers/merge_runs.py new file: .claude/skills/docx/scripts/office/helpers/simplify_redlines.py new file: .claude/skills/docx/scripts/office/pack.py new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd new file: .claude/skills/docx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd new file: .claude/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd new file: .claude/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd new file: .claude/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd new file: .claude/skills/docx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd new file: .claude/skills/docx/scripts/office/schemas/mce/mc.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-2010.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-2012.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-2018.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-cex-2018.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-cid-2016.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd new file: .claude/skills/docx/scripts/office/schemas/microsoft/wml-symex-2015.xsd new file: .claude/skills/docx/scripts/office/soffice.py new file: .claude/skills/docx/scripts/office/unpack.py new file: .claude/skills/docx/scripts/office/validate.py new file: .claude/skills/docx/scripts/office/validators/__init__.py new file: .claude/skills/docx/scripts/office/validators/base.py new file: .claude/skills/docx/scripts/office/validators/docx.py new file: .claude/skills/docx/scripts/office/validators/pptx.py new file: .claude/skills/docx/scripts/office/validators/redlining.py new file: .claude/skills/docx/scripts/templates/comments.xml new file: .claude/skills/docx/scripts/templates/commentsExtended.xml new file: .claude/skills/docx/scripts/templates/commentsExtensible.xml new file: .claude/skills/docx/scripts/templates/commentsIds.xml new file: .claude/skills/docx/scripts/templates/people.xml new file: .claude/skills/frontend-design/.openskills.json new file: .claude/skills/frontend-design/LICENSE.txt new file: .claude/skills/frontend-design/SKILL.md new file: .claude/skills/internal-comms/.openskills.json new file: .claude/skills/internal-comms/LICENSE.txt new file: .claude/skills/internal-comms/SKILL.md new file: .claude/skills/internal-comms/examples/3p-updates.md new file: .claude/skills/internal-comms/examples/company-newsletter.md new file: .claude/skills/internal-comms/examples/faq-answers.md new file: .claude/skills/internal-comms/examples/general-comms.md new file: .claude/skills/mcp-builder/.openskills.json new file: .claude/skills/mcp-builder/LICENSE.txt new file: .claude/skills/mcp-builder/SKILL.md new file: .claude/skills/mcp-builder/reference/evaluation.md new file: .claude/skills/mcp-builder/reference/mcp_best_practices.md new file: .claude/skills/mcp-builder/reference/node_mcp_server.md new file: .claude/skills/mcp-builder/reference/python_mcp_server.md new file: .claude/skills/mcp-builder/scripts/connections.py new file: .claude/skills/mcp-builder/scripts/evaluation.py new file: .claude/skills/mcp-builder/scripts/example_evaluation.xml new file: .claude/skills/mcp-builder/scripts/requirements.txt new file: .claude/skills/pdf/.openskills.json new file: .claude/skills/pdf/LICENSE.txt new file: .claude/skills/pdf/SKILL.md new file: .claude/skills/pdf/forms.md new file: .claude/skills/pdf/reference.md new file: .claude/skills/pdf/scripts/check_bounding_boxes.py new file: .claude/skills/pdf/scripts/check_fillable_fields.py new file: .claude/skills/pdf/scripts/convert_pdf_to_images.py new file: .claude/skills/pdf/scripts/create_validation_image.py new file: .claude/skills/pdf/scripts/extract_form_field_info.py new file: .claude/skills/pdf/scripts/extract_form_structure.py new file: .claude/skills/pdf/scripts/fill_fillable_fields.py new file: .claude/skills/pdf/scripts/fill_pdf_form_with_annotations.py new file: .claude/skills/pptx/.openskills.json new file: .claude/skills/pptx/LICENSE.txt new file: .claude/skills/pptx/SKILL.md new file: .claude/skills/pptx/editing.md new file: .claude/skills/pptx/pptxgenjs.md new file: .claude/skills/pptx/scripts/__init__.py new file: .claude/skills/pptx/scripts/add_slide.py new file: .claude/skills/pptx/scripts/clean.py new file: .claude/skills/pptx/scripts/office/helpers/__init__.py new file: .claude/skills/pptx/scripts/office/helpers/merge_runs.py new file: .claude/skills/pptx/scripts/office/helpers/simplify_redlines.py new file: .claude/skills/pptx/scripts/office/pack.py new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd new file: .claude/skills/pptx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd new file: .claude/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd new file: .claude/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd new file: .claude/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd new file: .claude/skills/pptx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd new file: .claude/skills/pptx/scripts/office/schemas/mce/mc.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-2010.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-2012.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-2018.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-cex-2018.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-cid-2016.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd new file: .claude/skills/pptx/scripts/office/schemas/microsoft/wml-symex-2015.xsd new file: .claude/skills/pptx/scripts/office/soffice.py new file: .claude/skills/pptx/scripts/office/unpack.py new file: .claude/skills/pptx/scripts/office/validate.py new file: .claude/skills/pptx/scripts/office/validators/__init__.py new file: .claude/skills/pptx/scripts/office/validators/base.py new file: .claude/skills/pptx/scripts/office/validators/docx.py new file: .claude/skills/pptx/scripts/office/validators/pptx.py new file: .claude/skills/pptx/scripts/office/validators/redlining.py new file: .claude/skills/pptx/scripts/thumbnail.py new file: .claude/skills/skill-creator/.openskills.json new file: .claude/skills/skill-creator/LICENSE.txt new file: .claude/skills/skill-creator/SKILL.md new file: .claude/skills/skill-creator/agents/analyzer.md new file: .claude/skills/skill-creator/agents/comparator.md new file: .claude/skills/skill-creator/agents/grader.md new file: .claude/skills/skill-creator/assets/eval_review.html new file: .claude/skills/skill-creator/eval-viewer/generate_review.py new file: .claude/skills/skill-creator/eval-viewer/viewer.html new file: .claude/skills/skill-creator/references/schemas.md new file: .claude/skills/skill-creator/scripts/__init__.py new file: .claude/skills/skill-creator/scripts/aggregate_benchmark.py new file: .claude/skills/skill-creator/scripts/generate_report.py new file: .claude/skills/skill-creator/scripts/improve_description.py new file: .claude/skills/skill-creator/scripts/package_skill.py new file: .claude/skills/skill-creator/scripts/quick_validate.py new file: .claude/skills/skill-creator/scripts/run_eval.py new file: .claude/skills/skill-creator/scripts/run_loop.py new file: .claude/skills/skill-creator/scripts/utils.py new file: .claude/skills/slack-gif-creator/.openskills.json new file: .claude/skills/slack-gif-creator/LICENSE.txt new file: .claude/skills/slack-gif-creator/SKILL.md new file: .claude/skills/slack-gif-creator/core/easing.py new file: .claude/skills/slack-gif-creator/core/frame_composer.py new file: .claude/skills/slack-gif-creator/core/gif_builder.py new file: .claude/skills/slack-gif-creator/core/validators.py new file: .claude/skills/slack-gif-creator/requirements.txt new file: .claude/skills/template/.openskills.json new file: .claude/skills/template/SKILL.md new file: .claude/skills/theme-factory/.openskills.json new file: .claude/skills/theme-factory/LICENSE.txt new file: .claude/skills/theme-factory/SKILL.md new file: .claude/skills/theme-factory/theme-showcase.pdf new file: .claude/skills/theme-factory/themes/arctic-frost.md new file: .claude/skills/theme-factory/themes/botanical-garden.md new file: .claude/skills/theme-factory/themes/desert-rose.md new file: .claude/skills/theme-factory/themes/forest-canopy.md new file: .claude/skills/theme-factory/themes/golden-hour.md new file: .claude/skills/theme-factory/themes/midnight-galaxy.md new file: .claude/skills/theme-factory/themes/modern-minimalist.md new file: .claude/skills/theme-factory/themes/ocean-depths.md new file: .claude/skills/theme-factory/themes/sunset-boulevard.md new file: .claude/skills/theme-factory/themes/tech-innovation.md new file: .claude/skills/web-artifacts-builder/.openskills.json new file: .claude/skills/web-artifacts-builder/LICENSE.txt new file: .claude/skills/web-artifacts-builder/SKILL.md new file: .claude/skills/web-artifacts-builder/scripts/bundle-artifact.sh new file: .claude/skills/web-artifacts-builder/scripts/init-artifact.sh new file: .claude/skills/web-artifacts-builder/scripts/shadcn-components.tar.gz new file: .claude/skills/webapp-testing/.openskills.json new file: .claude/skills/webapp-testing/LICENSE.txt new file: .claude/skills/webapp-testing/SKILL.md new file: .claude/skills/webapp-testing/examples/console_logging.py new file: .claude/skills/webapp-testing/examples/element_discovery.py new file: .claude/skills/webapp-testing/examples/static_html_automation.py new file: .claude/skills/webapp-testing/scripts/with_server.py new file: .claude/skills/xlsx/.openskills.json new file: .claude/skills/xlsx/LICENSE.txt new file: .claude/skills/xlsx/SKILL.md new file: .claude/skills/xlsx/scripts/office/helpers/__init__.py new file: .claude/skills/xlsx/scripts/office/helpers/merge_runs.py new file: .claude/skills/xlsx/scripts/office/helpers/simplify_redlines.py new file: .claude/skills/xlsx/scripts/office/pack.py new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chart.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-chartDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-diagram.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-lockedCanvas.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-main.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-picture.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-spreadsheetDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/dml-wordprocessingDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/pml.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-additionalCharacteristics.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-bibliography.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-commonSimpleTypes.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlDataProperties.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-customXmlSchemaProperties.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesCustom.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesExtended.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-documentPropertiesVariantTypes.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-math.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/shared-relationshipReference.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/sml.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-main.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-officeDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-presentationDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-spreadsheetDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/vml-wordprocessingDrawing.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/wml.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ISO-IEC29500-4_2016/xml.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-contentTypes.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-coreProperties.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-digSig.xsd new file: .claude/skills/xlsx/scripts/office/schemas/ecma/fouth-edition/opc-relationships.xsd new file: .claude/skills/xlsx/scripts/office/schemas/mce/mc.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-2010.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-2012.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-2018.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-cex-2018.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-cid-2016.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-sdtdatahash-2020.xsd new file: .claude/skills/xlsx/scripts/office/schemas/microsoft/wml-symex-2015.xsd new file: .claude/skills/xlsx/scripts/office/soffice.py new file: .claude/skills/xlsx/scripts/office/unpack.py new file: .claude/skills/xlsx/scripts/office/validate.py new file: .claude/skills/xlsx/scripts/office/validators/__init__.py new file: .claude/skills/xlsx/scripts/office/validators/base.py new file: .claude/skills/xlsx/scripts/office/validators/docx.py new file: .claude/skills/xlsx/scripts/office/validators/pptx.py new file: .claude/skills/xlsx/scripts/office/validators/redlining.py new file: .claude/skills/xlsx/scripts/recalc.py new file: .env.example new file: .gitignore new file: config/mcp.json new file: config/models.json new file: config/personalities.json new file: docs/AGENTS.md new file: docs/AI_IMPLEMENTATION.md new file: docs/AI_INTEGRATION_COMPLETE.md new file: docs/AI_QUICKSTART.md new file: docs/AI_SUMMARY.md new file: docs/CHANGELOG.md new file: docs/CONFIG_GUIDE.md new file: docs/FIXES.md new file: docs/PROJECT_REFACTOR.md new file: docs/README.md new file: docs/README_INDEX.md new file: examples/ai_example.py new file: main.py new file: pytest.ini new file: requirements.txt new file: scripts/migrate_to_vector_db.py new file: skills/cmd_zip_skill/README.md new file: skills/cmd_zip_skill/__init__.py new file: skills/cmd_zip_skill/main.py new file: skills/cmd_zip_skill/skill.json new file: skills/cmd_zip_skill_1772465404375/README.md new file: skills/cmd_zip_skill_1772465404375/__init__.py new file: skills/cmd_zip_skill_1772465404375/main.py new file: skills/cmd_zip_skill_1772465404375/skill.json new file: skills/cmd_zip_skill_1772465434774/README.md new file: skills/cmd_zip_skill_1772465434774/__init__.py new file: skills/cmd_zip_skill_1772465434774/main.py new file: skills/cmd_zip_skill_1772465434774/skill.json new file: skills/cmd_zip_skill_1772465467809/README.md new file: skills/cmd_zip_skill_1772465467809/__init__.py new file: skills/cmd_zip_skill_1772465467809/main.py new file: skills/cmd_zip_skill_1772465467809/skill.json new file: skills/cmd_zip_skill_1772465652075/README.md new file: skills/cmd_zip_skill_1772465652075/__init__.py new file: skills/cmd_zip_skill_1772465652075/main.py new file: skills/cmd_zip_skill_1772465652075/skill.json new file: skills/cmd_zip_skill_1772465685352/README.md new file: skills/cmd_zip_skill_1772465685352/__init__.py new file: skills/cmd_zip_skill_1772465685352/main.py new file: skills/cmd_zip_skill_1772465685352/skill.json new file: skills/cmd_zip_skill_1772465936294/README.md new file: skills/cmd_zip_skill_1772465936294/__init__.py new file: skills/cmd_zip_skill_1772465936294/main.py new file: skills/cmd_zip_skill_1772465936294/skill.json new file: skills/cmd_zip_skill_1772465966322/README.md new file: skills/cmd_zip_skill_1772465966322/__init__.py new file: skills/cmd_zip_skill_1772465966322/main.py new file: skills/cmd_zip_skill_1772465966322/skill.json new file: skills/cmd_zip_skill_1772466071278/README.md new file: skills/cmd_zip_skill_1772466071278/__init__.py new file: skills/cmd_zip_skill_1772466071278/main.py new file: skills/cmd_zip_skill_1772466071278/skill.json new file: skills/skills_creator/README.md new file: skills/skills_creator/__init__.py new file: skills/skills_creator/main.py new file: skills/skills_creator/skill.json new file: src/__init__.py new file: src/ai/__init__.py new file: src/ai/base.py new file: src/ai/client.py new file: src/ai/docs/README.md new file: src/ai/mcp/__init__.py new file: src/ai/mcp/base.py new file: src/ai/mcp/servers/__init__.py new file: src/ai/mcp/servers/filesystem.py new file: src/ai/memory.py new file: src/ai/models/__init__.py new file: src/ai/models/anthropic_model.py new file: src/ai/models/openai_model.py new file: src/ai/personality.py new file: src/ai/skills/__init__.py new file: src/ai/skills/base.py new file: src/ai/task_manager.py new file: src/ai/vector_store/__init__.py new file: src/ai/vector_store/base.py new file: src/ai/vector_store/chroma_store.py new file: src/ai/vector_store/json_store.py new file: src/core/__init__.py new file: src/core/bot.py new file: src/core/config.py new file: src/handlers/__init__.py new file: src/handlers/message_handler.py new file: src/handlers/message_handler_ai.py new file: src/utils/__init__.py new file: src/utils/logger.py new file: start.bat new file: tests/test_ai.py
This commit is contained in:
Mimikko-zeus
274
.claude/skills/skill-creator/agents/analyzer.md
Normal file
274
.claude/skills/skill-creator/agents/analyzer.md
Normal file
@@ -0,0 +1,274 @@
|
||||
# Post-hoc Analyzer Agent
|
||||
|
||||
Analyze blind comparison results to understand WHY the winner won and generate improvement suggestions.
|
||||
|
||||
## Role
|
||||
|
||||
After the blind comparator determines a winner, the Post-hoc Analyzer "unblids" the results by examining the skills and transcripts. The goal is to extract actionable insights: what made the winner better, and how can the loser be improved?
|
||||
|
||||
## Inputs
|
||||
|
||||
You receive these parameters in your prompt:
|
||||
|
||||
- **winner**: "A" or "B" (from blind comparison)
|
||||
- **winner_skill_path**: Path to the skill that produced the winning output
|
||||
- **winner_transcript_path**: Path to the execution transcript for the winner
|
||||
- **loser_skill_path**: Path to the skill that produced the losing output
|
||||
- **loser_transcript_path**: Path to the execution transcript for the loser
|
||||
- **comparison_result_path**: Path to the blind comparator's output JSON
|
||||
- **output_path**: Where to save the analysis results
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Read Comparison Result
|
||||
|
||||
1. Read the blind comparator's output at comparison_result_path
|
||||
2. Note the winning side (A or B), the reasoning, and any scores
|
||||
3. Understand what the comparator valued in the winning output
|
||||
|
||||
### Step 2: Read Both Skills
|
||||
|
||||
1. Read the winner skill's SKILL.md and key referenced files
|
||||
2. Read the loser skill's SKILL.md and key referenced files
|
||||
3. Identify structural differences:
|
||||
- Instructions clarity and specificity
|
||||
- Script/tool usage patterns
|
||||
- Example coverage
|
||||
- Edge case handling
|
||||
|
||||
### Step 3: Read Both Transcripts
|
||||
|
||||
1. Read the winner's transcript
|
||||
2. Read the loser's transcript
|
||||
3. Compare execution patterns:
|
||||
- How closely did each follow their skill's instructions?
|
||||
- What tools were used differently?
|
||||
- Where did the loser diverge from optimal behavior?
|
||||
- Did either encounter errors or make recovery attempts?
|
||||
|
||||
### Step 4: Analyze Instruction Following
|
||||
|
||||
For each transcript, evaluate:
|
||||
- Did the agent follow the skill's explicit instructions?
|
||||
- Did the agent use the skill's provided tools/scripts?
|
||||
- Were there missed opportunities to leverage skill content?
|
||||
- Did the agent add unnecessary steps not in the skill?
|
||||
|
||||
Score instruction following 1-10 and note specific issues.
|
||||
|
||||
### Step 5: Identify Winner Strengths
|
||||
|
||||
Determine what made the winner better:
|
||||
- Clearer instructions that led to better behavior?
|
||||
- Better scripts/tools that produced better output?
|
||||
- More comprehensive examples that guided edge cases?
|
||||
- Better error handling guidance?
|
||||
|
||||
Be specific. Quote from skills/transcripts where relevant.
|
||||
|
||||
### Step 6: Identify Loser Weaknesses
|
||||
|
||||
Determine what held the loser back:
|
||||
- Ambiguous instructions that led to suboptimal choices?
|
||||
- Missing tools/scripts that forced workarounds?
|
||||
- Gaps in edge case coverage?
|
||||
- Poor error handling that caused failures?
|
||||
|
||||
### Step 7: Generate Improvement Suggestions
|
||||
|
||||
Based on the analysis, produce actionable suggestions for improving the loser skill:
|
||||
- Specific instruction changes to make
|
||||
- Tools/scripts to add or modify
|
||||
- Examples to include
|
||||
- Edge cases to address
|
||||
|
||||
Prioritize by impact. Focus on changes that would have changed the outcome.
|
||||
|
||||
### Step 8: Write Analysis Results
|
||||
|
||||
Save structured analysis to `{output_path}`.
|
||||
|
||||
## Output Format
|
||||
|
||||
Write a JSON file with this structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"comparison_summary": {
|
||||
"winner": "A",
|
||||
"winner_skill": "path/to/winner/skill",
|
||||
"loser_skill": "path/to/loser/skill",
|
||||
"comparator_reasoning": "Brief summary of why comparator chose winner"
|
||||
},
|
||||
"winner_strengths": [
|
||||
"Clear step-by-step instructions for handling multi-page documents",
|
||||
"Included validation script that caught formatting errors",
|
||||
"Explicit guidance on fallback behavior when OCR fails"
|
||||
],
|
||||
"loser_weaknesses": [
|
||||
"Vague instruction 'process the document appropriately' led to inconsistent behavior",
|
||||
"No script for validation, agent had to improvise and made errors",
|
||||
"No guidance on OCR failure, agent gave up instead of trying alternatives"
|
||||
],
|
||||
"instruction_following": {
|
||||
"winner": {
|
||||
"score": 9,
|
||||
"issues": [
|
||||
"Minor: skipped optional logging step"
|
||||
]
|
||||
},
|
||||
"loser": {
|
||||
"score": 6,
|
||||
"issues": [
|
||||
"Did not use the skill's formatting template",
|
||||
"Invented own approach instead of following step 3",
|
||||
"Missed the 'always validate output' instruction"
|
||||
]
|
||||
}
|
||||
},
|
||||
"improvement_suggestions": [
|
||||
{
|
||||
"priority": "high",
|
||||
"category": "instructions",
|
||||
"suggestion": "Replace 'process the document appropriately' with explicit steps: 1) Extract text, 2) Identify sections, 3) Format per template",
|
||||
"expected_impact": "Would eliminate ambiguity that caused inconsistent behavior"
|
||||
},
|
||||
{
|
||||
"priority": "high",
|
||||
"category": "tools",
|
||||
"suggestion": "Add validate_output.py script similar to winner skill's validation approach",
|
||||
"expected_impact": "Would catch formatting errors before final output"
|
||||
},
|
||||
{
|
||||
"priority": "medium",
|
||||
"category": "error_handling",
|
||||
"suggestion": "Add fallback instructions: 'If OCR fails, try: 1) different resolution, 2) image preprocessing, 3) manual extraction'",
|
||||
"expected_impact": "Would prevent early failure on difficult documents"
|
||||
}
|
||||
],
|
||||
"transcript_insights": {
|
||||
"winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script -> Fixed 2 issues -> Produced output",
|
||||
"loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods -> No validation -> Output had errors"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Be specific**: Quote from skills and transcripts, don't just say "instructions were unclear"
|
||||
- **Be actionable**: Suggestions should be concrete changes, not vague advice
|
||||
- **Focus on skill improvements**: The goal is to improve the losing skill, not critique the agent
|
||||
- **Prioritize by impact**: Which changes would most likely have changed the outcome?
|
||||
- **Consider causation**: Did the skill weakness actually cause the worse output, or is it incidental?
|
||||
- **Stay objective**: Analyze what happened, don't editorialize
|
||||
- **Think about generalization**: Would this improvement help on other evals too?
|
||||
|
||||
## Categories for Suggestions
|
||||
|
||||
Use these categories to organize improvement suggestions:
|
||||
|
||||
| Category | Description |
|
||||
|----------|-------------|
|
||||
| `instructions` | Changes to the skill's prose instructions |
|
||||
| `tools` | Scripts, templates, or utilities to add/modify |
|
||||
| `examples` | Example inputs/outputs to include |
|
||||
| `error_handling` | Guidance for handling failures |
|
||||
| `structure` | Reorganization of skill content |
|
||||
| `references` | External docs or resources to add |
|
||||
|
||||
## Priority Levels
|
||||
|
||||
- **high**: Would likely change the outcome of this comparison
|
||||
- **medium**: Would improve quality but may not change win/loss
|
||||
- **low**: Nice to have, marginal improvement
|
||||
|
||||
---
|
||||
|
||||
# Analyzing Benchmark Results
|
||||
|
||||
When analyzing benchmark results, the analyzer's purpose is to **surface patterns and anomalies** across multiple runs, not suggest skill improvements.
|
||||
|
||||
## Role
|
||||
|
||||
Review all benchmark run results and generate freeform notes that help the user understand skill performance. Focus on patterns that wouldn't be visible from aggregate metrics alone.
|
||||
|
||||
## Inputs
|
||||
|
||||
You receive these parameters in your prompt:
|
||||
|
||||
- **benchmark_data_path**: Path to the in-progress benchmark.json with all run results
|
||||
- **skill_path**: Path to the skill being benchmarked
|
||||
- **output_path**: Where to save the notes (as JSON array of strings)
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Read Benchmark Data
|
||||
|
||||
1. Read the benchmark.json containing all run results
|
||||
2. Note the configurations tested (with_skill, without_skill)
|
||||
3. Understand the run_summary aggregates already calculated
|
||||
|
||||
### Step 2: Analyze Per-Assertion Patterns
|
||||
|
||||
For each expectation across all runs:
|
||||
- Does it **always pass** in both configurations? (may not differentiate skill value)
|
||||
- Does it **always fail** in both configurations? (may be broken or beyond capability)
|
||||
- Does it **always pass with skill but fail without**? (skill clearly adds value here)
|
||||
- Does it **always fail with skill but pass without**? (skill may be hurting)
|
||||
- Is it **highly variable**? (flaky expectation or non-deterministic behavior)
|
||||
|
||||
### Step 3: Analyze Cross-Eval Patterns
|
||||
|
||||
Look for patterns across evals:
|
||||
- Are certain eval types consistently harder/easier?
|
||||
- Do some evals show high variance while others are stable?
|
||||
- Are there surprising results that contradict expectations?
|
||||
|
||||
### Step 4: Analyze Metrics Patterns
|
||||
|
||||
Look at time_seconds, tokens, tool_calls:
|
||||
- Does the skill significantly increase execution time?
|
||||
- Is there high variance in resource usage?
|
||||
- Are there outlier runs that skew the aggregates?
|
||||
|
||||
### Step 5: Generate Notes
|
||||
|
||||
Write freeform observations as a list of strings. Each note should:
|
||||
- State a specific observation
|
||||
- Be grounded in the data (not speculation)
|
||||
- Help the user understand something the aggregate metrics don't show
|
||||
|
||||
Examples:
|
||||
- "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value"
|
||||
- "Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure that may be flaky"
|
||||
- "Without-skill runs consistently fail on table extraction expectations (0% pass rate)"
|
||||
- "Skill adds 13s average execution time but improves pass rate by 50%"
|
||||
- "Token usage is 80% higher with skill, primarily due to script output parsing"
|
||||
- "All 3 without-skill runs for eval 1 produced empty output"
|
||||
|
||||
### Step 6: Write Notes
|
||||
|
||||
Save notes to `{output_path}` as a JSON array of strings:
|
||||
|
||||
```json
|
||||
[
|
||||
"Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value",
|
||||
"Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure",
|
||||
"Without-skill runs consistently fail on table extraction expectations",
|
||||
"Skill adds 13s average execution time but improves pass rate by 50%"
|
||||
]
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
**DO:**
|
||||
- Report what you observe in the data
|
||||
- Be specific about which evals, expectations, or runs you're referring to
|
||||
- Note patterns that aggregate metrics would hide
|
||||
- Provide context that helps interpret the numbers
|
||||
|
||||
**DO NOT:**
|
||||
- Suggest improvements to the skill (that's for the improvement step, not benchmarking)
|
||||
- Make subjective quality judgments ("the output was good/bad")
|
||||
- Speculate about causes without evidence
|
||||
- Repeat information already in the run_summary aggregates
|
||||
202
.claude/skills/skill-creator/agents/comparator.md
Normal file
202
.claude/skills/skill-creator/agents/comparator.md
Normal file
@@ -0,0 +1,202 @@
|
||||
# Blind Comparator Agent
|
||||
|
||||
Compare two outputs WITHOUT knowing which skill produced them.
|
||||
|
||||
## Role
|
||||
|
||||
The Blind Comparator judges which output better accomplishes the eval task. You receive two outputs labeled A and B, but you do NOT know which skill produced which. This prevents bias toward a particular skill or approach.
|
||||
|
||||
Your judgment is based purely on output quality and task completion.
|
||||
|
||||
## Inputs
|
||||
|
||||
You receive these parameters in your prompt:
|
||||
|
||||
- **output_a_path**: Path to the first output file or directory
|
||||
- **output_b_path**: Path to the second output file or directory
|
||||
- **eval_prompt**: The original task/prompt that was executed
|
||||
- **expectations**: List of expectations to check (optional - may be empty)
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Read Both Outputs
|
||||
|
||||
1. Examine output A (file or directory)
|
||||
2. Examine output B (file or directory)
|
||||
3. Note the type, structure, and content of each
|
||||
4. If outputs are directories, examine all relevant files inside
|
||||
|
||||
### Step 2: Understand the Task
|
||||
|
||||
1. Read the eval_prompt carefully
|
||||
2. Identify what the task requires:
|
||||
- What should be produced?
|
||||
- What qualities matter (accuracy, completeness, format)?
|
||||
- What would distinguish a good output from a poor one?
|
||||
|
||||
### Step 3: Generate Evaluation Rubric
|
||||
|
||||
Based on the task, generate a rubric with two dimensions:
|
||||
|
||||
**Content Rubric** (what the output contains):
|
||||
| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) |
|
||||
|-----------|----------|----------------|---------------|
|
||||
| Correctness | Major errors | Minor errors | Fully correct |
|
||||
| Completeness | Missing key elements | Mostly complete | All elements present |
|
||||
| Accuracy | Significant inaccuracies | Minor inaccuracies | Accurate throughout |
|
||||
|
||||
**Structure Rubric** (how the output is organized):
|
||||
| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) |
|
||||
|-----------|----------|----------------|---------------|
|
||||
| Organization | Disorganized | Reasonably organized | Clear, logical structure |
|
||||
| Formatting | Inconsistent/broken | Mostly consistent | Professional, polished |
|
||||
| Usability | Difficult to use | Usable with effort | Easy to use |
|
||||
|
||||
Adapt criteria to the specific task. For example:
|
||||
- PDF form → "Field alignment", "Text readability", "Data placement"
|
||||
- Document → "Section structure", "Heading hierarchy", "Paragraph flow"
|
||||
- Data output → "Schema correctness", "Data types", "Completeness"
|
||||
|
||||
### Step 4: Evaluate Each Output Against the Rubric
|
||||
|
||||
For each output (A and B):
|
||||
|
||||
1. **Score each criterion** on the rubric (1-5 scale)
|
||||
2. **Calculate dimension totals**: Content score, Structure score
|
||||
3. **Calculate overall score**: Average of dimension scores, scaled to 1-10
|
||||
|
||||
### Step 5: Check Assertions (if provided)
|
||||
|
||||
If expectations are provided:
|
||||
|
||||
1. Check each expectation against output A
|
||||
2. Check each expectation against output B
|
||||
3. Count pass rates for each output
|
||||
4. Use expectation scores as secondary evidence (not the primary decision factor)
|
||||
|
||||
### Step 6: Determine the Winner
|
||||
|
||||
Compare A and B based on (in priority order):
|
||||
|
||||
1. **Primary**: Overall rubric score (content + structure)
|
||||
2. **Secondary**: Assertion pass rates (if applicable)
|
||||
3. **Tiebreaker**: If truly equal, declare a TIE
|
||||
|
||||
Be decisive - ties should be rare. One output is usually better, even if marginally.
|
||||
|
||||
### Step 7: Write Comparison Results
|
||||
|
||||
Save results to a JSON file at the path specified (or `comparison.json` if not specified).
|
||||
|
||||
## Output Format
|
||||
|
||||
Write a JSON file with this structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"winner": "A",
|
||||
"reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.",
|
||||
"rubric": {
|
||||
"A": {
|
||||
"content": {
|
||||
"correctness": 5,
|
||||
"completeness": 5,
|
||||
"accuracy": 4
|
||||
},
|
||||
"structure": {
|
||||
"organization": 4,
|
||||
"formatting": 5,
|
||||
"usability": 4
|
||||
},
|
||||
"content_score": 4.7,
|
||||
"structure_score": 4.3,
|
||||
"overall_score": 9.0
|
||||
},
|
||||
"B": {
|
||||
"content": {
|
||||
"correctness": 3,
|
||||
"completeness": 2,
|
||||
"accuracy": 3
|
||||
},
|
||||
"structure": {
|
||||
"organization": 3,
|
||||
"formatting": 2,
|
||||
"usability": 3
|
||||
},
|
||||
"content_score": 2.7,
|
||||
"structure_score": 2.7,
|
||||
"overall_score": 5.4
|
||||
}
|
||||
},
|
||||
"output_quality": {
|
||||
"A": {
|
||||
"score": 9,
|
||||
"strengths": ["Complete solution", "Well-formatted", "All fields present"],
|
||||
"weaknesses": ["Minor style inconsistency in header"]
|
||||
},
|
||||
"B": {
|
||||
"score": 5,
|
||||
"strengths": ["Readable output", "Correct basic structure"],
|
||||
"weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"]
|
||||
}
|
||||
},
|
||||
"expectation_results": {
|
||||
"A": {
|
||||
"passed": 4,
|
||||
"total": 5,
|
||||
"pass_rate": 0.80,
|
||||
"details": [
|
||||
{"text": "Output includes name", "passed": true},
|
||||
{"text": "Output includes date", "passed": true},
|
||||
{"text": "Format is PDF", "passed": true},
|
||||
{"text": "Contains signature", "passed": false},
|
||||
{"text": "Readable text", "passed": true}
|
||||
]
|
||||
},
|
||||
"B": {
|
||||
"passed": 3,
|
||||
"total": 5,
|
||||
"pass_rate": 0.60,
|
||||
"details": [
|
||||
{"text": "Output includes name", "passed": true},
|
||||
{"text": "Output includes date", "passed": false},
|
||||
{"text": "Format is PDF", "passed": true},
|
||||
{"text": "Contains signature", "passed": false},
|
||||
{"text": "Readable text", "passed": true}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If no expectations were provided, omit the `expectation_results` field entirely.
|
||||
|
||||
## Field Descriptions
|
||||
|
||||
- **winner**: "A", "B", or "TIE"
|
||||
- **reasoning**: Clear explanation of why the winner was chosen (or why it's a tie)
|
||||
- **rubric**: Structured rubric evaluation for each output
|
||||
- **content**: Scores for content criteria (correctness, completeness, accuracy)
|
||||
- **structure**: Scores for structure criteria (organization, formatting, usability)
|
||||
- **content_score**: Average of content criteria (1-5)
|
||||
- **structure_score**: Average of structure criteria (1-5)
|
||||
- **overall_score**: Combined score scaled to 1-10
|
||||
- **output_quality**: Summary quality assessment
|
||||
- **score**: 1-10 rating (should match rubric overall_score)
|
||||
- **strengths**: List of positive aspects
|
||||
- **weaknesses**: List of issues or shortcomings
|
||||
- **expectation_results**: (Only if expectations provided)
|
||||
- **passed**: Number of expectations that passed
|
||||
- **total**: Total number of expectations
|
||||
- **pass_rate**: Fraction passed (0.0 to 1.0)
|
||||
- **details**: Individual expectation results
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Stay blind**: DO NOT try to infer which skill produced which output. Judge purely on output quality.
|
||||
- **Be specific**: Cite specific examples when explaining strengths and weaknesses.
|
||||
- **Be decisive**: Choose a winner unless outputs are genuinely equivalent.
|
||||
- **Output quality first**: Assertion scores are secondary to overall task completion.
|
||||
- **Be objective**: Don't favor outputs based on style preferences; focus on correctness and completeness.
|
||||
- **Explain your reasoning**: The reasoning field should make it clear why you chose the winner.
|
||||
- **Handle edge cases**: If both outputs fail, pick the one that fails less badly. If both are excellent, pick the one that's marginally better.
|
||||
223
.claude/skills/skill-creator/agents/grader.md
Normal file
223
.claude/skills/skill-creator/agents/grader.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Grader Agent
|
||||
|
||||
Evaluate expectations against an execution transcript and outputs.
|
||||
|
||||
## Role
|
||||
|
||||
The Grader reviews a transcript and output files, then determines whether each expectation passes or fails. Provide clear evidence for each judgment.
|
||||
|
||||
You have two jobs: grade the outputs, and critique the evals themselves. A passing grade on a weak assertion is worse than useless — it creates false confidence. When you notice an assertion that's trivially satisfied, or an important outcome that no assertion checks, say so.
|
||||
|
||||
## Inputs
|
||||
|
||||
You receive these parameters in your prompt:
|
||||
|
||||
- **expectations**: List of expectations to evaluate (strings)
|
||||
- **transcript_path**: Path to the execution transcript (markdown file)
|
||||
- **outputs_dir**: Directory containing output files from execution
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Read the Transcript
|
||||
|
||||
1. Read the transcript file completely
|
||||
2. Note the eval prompt, execution steps, and final result
|
||||
3. Identify any issues or errors documented
|
||||
|
||||
### Step 2: Examine Output Files
|
||||
|
||||
1. List files in outputs_dir
|
||||
2. Read/examine each file relevant to the expectations. If outputs aren't plain text, use the inspection tools provided in your prompt — don't rely solely on what the transcript says the executor produced.
|
||||
3. Note contents, structure, and quality
|
||||
|
||||
### Step 3: Evaluate Each Assertion
|
||||
|
||||
For each expectation:
|
||||
|
||||
1. **Search for evidence** in the transcript and outputs
|
||||
2. **Determine verdict**:
|
||||
- **PASS**: Clear evidence the expectation is true AND the evidence reflects genuine task completion, not just surface-level compliance
|
||||
- **FAIL**: No evidence, or evidence contradicts the expectation, or the evidence is superficial (e.g., correct filename but empty/wrong content)
|
||||
3. **Cite the evidence**: Quote the specific text or describe what you found
|
||||
|
||||
### Step 4: Extract and Verify Claims
|
||||
|
||||
Beyond the predefined expectations, extract implicit claims from the outputs and verify them:
|
||||
|
||||
1. **Extract claims** from the transcript and outputs:
|
||||
- Factual statements ("The form has 12 fields")
|
||||
- Process claims ("Used pypdf to fill the form")
|
||||
- Quality claims ("All fields were filled correctly")
|
||||
|
||||
2. **Verify each claim**:
|
||||
- **Factual claims**: Can be checked against the outputs or external sources
|
||||
- **Process claims**: Can be verified from the transcript
|
||||
- **Quality claims**: Evaluate whether the claim is justified
|
||||
|
||||
3. **Flag unverifiable claims**: Note claims that cannot be verified with available information
|
||||
|
||||
This catches issues that predefined expectations might miss.
|
||||
|
||||
### Step 5: Read User Notes
|
||||
|
||||
If `{outputs_dir}/user_notes.md` exists:
|
||||
1. Read it and note any uncertainties or issues flagged by the executor
|
||||
2. Include relevant concerns in the grading output
|
||||
3. These may reveal problems even when expectations pass
|
||||
|
||||
### Step 6: Critique the Evals
|
||||
|
||||
After grading, consider whether the evals themselves could be improved. Only surface suggestions when there's a clear gap.
|
||||
|
||||
Good suggestions test meaningful outcomes — assertions that are hard to satisfy without actually doing the work correctly. Think about what makes an assertion *discriminating*: it passes when the skill genuinely succeeds and fails when it doesn't.
|
||||
|
||||
Suggestions worth raising:
|
||||
- An assertion that passed but would also pass for a clearly wrong output (e.g., checking filename existence but not file content)
|
||||
- An important outcome you observed — good or bad — that no assertion covers at all
|
||||
- An assertion that can't actually be verified from the available outputs
|
||||
|
||||
Keep the bar high. The goal is to flag things the eval author would say "good catch" about, not to nitpick every assertion.
|
||||
|
||||
### Step 7: Write Grading Results
|
||||
|
||||
Save results to `{outputs_dir}/../grading.json` (sibling to outputs_dir).
|
||||
|
||||
## Grading Criteria
|
||||
|
||||
**PASS when**:
|
||||
- The transcript or outputs clearly demonstrate the expectation is true
|
||||
- Specific evidence can be cited
|
||||
- The evidence reflects genuine substance, not just surface compliance (e.g., a file exists AND contains correct content, not just the right filename)
|
||||
|
||||
**FAIL when**:
|
||||
- No evidence found for the expectation
|
||||
- Evidence contradicts the expectation
|
||||
- The expectation cannot be verified from available information
|
||||
- The evidence is superficial — the assertion is technically satisfied but the underlying task outcome is wrong or incomplete
|
||||
- The output appears to meet the assertion by coincidence rather than by actually doing the work
|
||||
|
||||
**When uncertain**: The burden of proof to pass is on the expectation.
|
||||
|
||||
### Step 8: Read Executor Metrics and Timing
|
||||
|
||||
1. If `{outputs_dir}/metrics.json` exists, read it and include in grading output
|
||||
2. If `{outputs_dir}/../timing.json` exists, read it and include timing data
|
||||
|
||||
## Output Format
|
||||
|
||||
Write a JSON file with this structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"expectations": [
|
||||
{
|
||||
"text": "The output includes the name 'John Smith'",
|
||||
"passed": true,
|
||||
"evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'"
|
||||
},
|
||||
{
|
||||
"text": "The spreadsheet has a SUM formula in cell B10",
|
||||
"passed": false,
|
||||
"evidence": "No spreadsheet was created. The output was a text file."
|
||||
},
|
||||
{
|
||||
"text": "The assistant used the skill's OCR script",
|
||||
"passed": true,
|
||||
"evidence": "Transcript Step 2 shows: 'Tool: Bash - python ocr_script.py image.png'"
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
"passed": 2,
|
||||
"failed": 1,
|
||||
"total": 3,
|
||||
"pass_rate": 0.67
|
||||
},
|
||||
"execution_metrics": {
|
||||
"tool_calls": {
|
||||
"Read": 5,
|
||||
"Write": 2,
|
||||
"Bash": 8
|
||||
},
|
||||
"total_tool_calls": 15,
|
||||
"total_steps": 6,
|
||||
"errors_encountered": 0,
|
||||
"output_chars": 12450,
|
||||
"transcript_chars": 3200
|
||||
},
|
||||
"timing": {
|
||||
"executor_duration_seconds": 165.0,
|
||||
"grader_duration_seconds": 26.0,
|
||||
"total_duration_seconds": 191.0
|
||||
},
|
||||
"claims": [
|
||||
{
|
||||
"claim": "The form has 12 fillable fields",
|
||||
"type": "factual",
|
||||
"verified": true,
|
||||
"evidence": "Counted 12 fields in field_info.json"
|
||||
},
|
||||
{
|
||||
"claim": "All required fields were populated",
|
||||
"type": "quality",
|
||||
"verified": false,
|
||||
"evidence": "Reference section was left blank despite data being available"
|
||||
}
|
||||
],
|
||||
"user_notes_summary": {
|
||||
"uncertainties": ["Used 2023 data, may be stale"],
|
||||
"needs_review": [],
|
||||
"workarounds": ["Fell back to text overlay for non-fillable fields"]
|
||||
},
|
||||
"eval_feedback": {
|
||||
"suggestions": [
|
||||
{
|
||||
"assertion": "The output includes the name 'John Smith'",
|
||||
"reason": "A hallucinated document that mentions the name would also pass — consider checking it appears as the primary contact with matching phone and email from the input"
|
||||
},
|
||||
{
|
||||
"reason": "No assertion checks whether the extracted phone numbers match the input — I observed incorrect numbers in the output that went uncaught"
|
||||
}
|
||||
],
|
||||
"overall": "Assertions check presence but not correctness. Consider adding content verification."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Field Descriptions
|
||||
|
||||
- **expectations**: Array of graded expectations
|
||||
- **text**: The original expectation text
|
||||
- **passed**: Boolean - true if expectation passes
|
||||
- **evidence**: Specific quote or description supporting the verdict
|
||||
- **summary**: Aggregate statistics
|
||||
- **passed**: Count of passed expectations
|
||||
- **failed**: Count of failed expectations
|
||||
- **total**: Total expectations evaluated
|
||||
- **pass_rate**: Fraction passed (0.0 to 1.0)
|
||||
- **execution_metrics**: Copied from executor's metrics.json (if available)
|
||||
- **output_chars**: Total character count of output files (proxy for tokens)
|
||||
- **transcript_chars**: Character count of transcript
|
||||
- **timing**: Wall clock timing from timing.json (if available)
|
||||
- **executor_duration_seconds**: Time spent in executor subagent
|
||||
- **total_duration_seconds**: Total elapsed time for the run
|
||||
- **claims**: Extracted and verified claims from the output
|
||||
- **claim**: The statement being verified
|
||||
- **type**: "factual", "process", or "quality"
|
||||
- **verified**: Boolean - whether the claim holds
|
||||
- **evidence**: Supporting or contradicting evidence
|
||||
- **user_notes_summary**: Issues flagged by the executor
|
||||
- **uncertainties**: Things the executor wasn't sure about
|
||||
- **needs_review**: Items requiring human attention
|
||||
- **workarounds**: Places where the skill didn't work as expected
|
||||
- **eval_feedback**: Improvement suggestions for the evals (only when warranted)
|
||||
- **suggestions**: List of concrete suggestions, each with a `reason` and optionally an `assertion` it relates to
|
||||
- **overall**: Brief assessment — can be "No suggestions, evals look solid" if nothing to flag
|
||||
|
||||
## Guidelines
|
||||
|
||||
- **Be objective**: Base verdicts on evidence, not assumptions
|
||||
- **Be specific**: Quote the exact text that supports your verdict
|
||||
- **Be thorough**: Check both transcript and output files
|
||||
- **Be consistent**: Apply the same standard to each expectation
|
||||
- **Explain failures**: Make it clear why evidence was insufficient
|
||||
- **No partial credit**: Each expectation is pass or fail, not partial
|
||||
Reference in New Issue
Block a user