Code Execution API

Overview

The Canvas MCP Code Execution API can reduce model-context use for bulk operations. Instead of returning every submission to the model, code executes locally and returns selected output.

Context Efficiency Comparison

Scenario: Grading a large set of Jupyter notebook submissions

Traditional Approach (Inefficient)

This traditional example loads all submissions into the model's context:

// Load ALL submissions into context
const submissions = await list_submissions({
  courseIdentifier: "60366",
  assignmentId: "123"
});
// Each returned submission consumes model context.

// Process each one (more tokens!)
for (const sub of submissions) {
  await grade_with_rubric({
    courseIdentifier: "60366",
    assignmentId: "123",
    userId: sub.userId,
    rubricAssessment: { ... }
  });
}

Why This Is Inefficient

• ✗ All 90 submissions loaded into the model's context
• ✗ Context use grows with returned submission data
• ✗ This example processes submissions sequentially
• ✗ Risk of hitting token limits
• ✗ Model cost depends on the client, model, and payload size

Code Execution Approach (Efficient)

The code execution API processes data locally:

import { bulkGrade } from './canvas/grading/bulkGrade';

await bulkGrade({
  courseIdentifier: "60366",
  assignmentId: "123",
  gradingFunction: (submission) => {
    // ⭐ This function runs LOCALLY
    // Per-submission processing stays local; return only selected output.

    const notebook = submission.attachments?.find(
      f => f.filename.endsWith('.ipynb')
    );

    if (!notebook) {
      console.log(`No notebook for user ${submission.userId}`);
      return null; // Skip this submission
    }

    // Download and analyze notebook (locally!)
    const analysis = analyzeNotebook(notebook.url);

    if (analysis.hasErrors) {
      return {
        points: 0,
        rubricAssessment: {
          "_8027": {
            points: 0,
            comments: `Found errors: ${analysis.errors.join(', ')}`
          }
        },
        comment: "Please fix errors and resubmit."
      };
    }

    return {
      points: 100,
      rubricAssessment: {
        "_8027": { points: 100, comments: "Excellent work!" }
      },
      comment: "Great submission!"
    };
  }
});

Why This Is Efficient

• ✓ Only selected output needs to return to the model
• ✓ Data processed locally in execution environment
• ✓ Supported operations can process concurrently
• ✓ Context use depends on the code, selected output, and AI client
• ✓ Scale remains subject to Canvas rate limits and local resources

Bulk Grading Example

Output Format

Starting bulk grading for assignment 123...
Found 90 submissions to process

✓ Graded submission for user 12345
✓ Graded submission for user 12346
Skipped submission for user 12347 (no notebook)
✓ Graded submission for user 12348
✗ Failed to grade user 12349: Network timeout
...

Bulk grading complete:
  Total: 90
  Graded: 87
  Skipped: 2
  Failed: 1

Advanced Custom Analysis

await bulkGrade({
  courseIdentifier: "60366",
  assignmentId: "123",
  gradingFunction: (submission) => {
    const notebook = submission.attachments?.find(
      f => f.filename.endsWith('.ipynb')
    );

    if (!notebook) return null;

    // Custom analysis logic
    const analysis = {
      cellCount: countCells(notebook),
      hasDocstrings: checkDocstrings(notebook),
      passesTests: runTests(notebook),
      codeQuality: analyzeCodeQuality(notebook)
    };

    // Complex grading rubric
    let points = 0;
    const rubricComments = {};

    // Criterion 1: Functionality (50 points)
    if (analysis.passesTests) {
      points += 50;
      rubricComments["_8027"] = {
        points: 50,
        comments: "All tests pass!"
      };
    }

    // Criterion 2: Documentation (30 points)
    const docPoints = analysis.hasDocstrings ? 30 : 15;
    points += docPoints;

    // Criterion 3: Code Quality (20 points)
    const qualityPoints = Math.min(20, analysis.codeQuality * 20);
    points += qualityPoints;

    return { points, rubricAssessment: rubricComments };
  }
});

Bulk Discussion Grading

Grade discussion posts with initial post + peer review requirements:

import { bulkGradeDiscussion } from './canvas/discussions/bulkGradeDiscussion';

// Preview grades first (dry run)
await bulkGradeDiscussion({
  courseIdentifier: "60365",
  topicId: "990001",
  criteria: {
    initialPostPoints: 10,      // Points for initial post
    peerReviewPointsEach: 5,    // Points per peer review
    requiredPeerReviews: 2,     // Must review 2 peers
    maxPeerReviewPoints: 10     // Cap at 10 pts for reviews
  },
  dryRun: true  // Preview first!
});

// Then apply grades
await bulkGradeDiscussion({
  courseIdentifier: "60365",
  topicId: "990001",
  assignmentId: "1234567",  // Required to write grades
  criteria: { ... },
  dryRun: false
});

Features

• Automatically analyzes initial posts vs peer reviews
• Configurable grading criteria with point allocation
• Optional late penalties with customizable deadline
• Dry run mode to preview grades before applying
• Concurrent processing with rate limiting

Discovering Available Tools

Use the search_canvas_tools MCP tool to discover available operations:

// Search for grading-related tools
search_canvas_tools("grading", "signatures")

// List all available tools
search_canvas_tools("", "names")

// Get full implementation details
search_canvas_tools("bulk", "full")

Natural Language Discovery

Ask your AI assistant:

• "Search for grading tools in the code API"
• "What bulk operations are available?"
• "Show me all code API tools"

Code API File Structure

src/canvas_mcp/code_api/
├── client.ts              # Base MCP client bridge
├── index.ts               # Main entry point
└── canvas/
    ├── assignments/       # Assignment operations
    ├── grading/          # Grading operations
    │   ├── gradeWithRubric.ts
    │   └── bulkGrade.ts  # ⭐ Bulk grading
    ├── discussions/      # Discussion operations
    │   └── bulkGradeDiscussion.ts
    ├── courses/          # Course operations
    └── communications/   # Messaging operations

Dry Run Mode (Testing)

Always test your grading logic before actually grading:

await bulkGrade({
  courseIdentifier: "60366",
  assignmentId: "123",
  dryRun: true,  // ⭐ Test mode - doesn't actually grade
  gradingFunction: (submission) => {
    console.log(`Would grade: ${submission.userId}`);
    return { points: 100, ... };
  }
});

Best Practices

1. Always test with dry run first before grading for real
2. Handle errors gracefully - return null to skip problematic submissions
3. Provide detailed rubric comments to help students understand their grades
4. Log progress using console.log() to track grading status
5. Validate rubric criterion IDs before grading

Common Rubric Criterion ID Patterns

Canvas rubric criterion IDs typically start with underscore:

• "_8027" - Common format
• "criterion_123" - Alternative format
• "8027" - Without underscore (rare)

Troubleshooting

"No exported function found"

• Check that your TypeScript files have export async function declarations
• Verify file paths are correct

"Criterion ID not found"

• Use get_rubric (with the assignment ID) to get correct criterion IDs
• Remember: IDs often start with underscore ("_8027")

"Rate limit exceeded"

• Add delays between grading operations
• Reduce maxConcurrent parameter (default: 5)

"Submission not found"

• Check that courseIdentifier and assignmentId are correct
• Verify students have actually submitted