# Building a real-time streaming task manager with Parallel

Build a task streaming playground that showcases Parallel's Task API with real-time Server-Sent Events (SSE).

Reading time: 5 min

Building a real-time streaming task manager with Parallel

This guide demonstrates how to build a complete task streaming playground that showcases Parallel's Task API[Parallel's Task API]($https://docs.parallel.ai/task-api/task-quickstart) with real-time Server-Sent Events (SSE). By the end, you'll have a full-featured application that creates tasks, streams their execution progress, and displays results as they arrive. This can be helpful for developers building with the Task API, demonstrating how to recreate the user interfaces in our Playground[Playground]($https://platform.parallel.ai).

Complete demo available at: https://oss.parallel.ai/tasks-sse/[https://oss.parallel.ai/tasks-sse/]($https://oss.parallel.ai/tasks-sse/)

Illustration demonstrating deep research API concepts, web search capabilities, or AI agent integration features

## Key Features

- Task Manager for All Processors: From lite ($5/1K) to ultra8x ($2400/1K)
- Output Schemas: Text, JSON and Auto Schema
- Source Policy: Domain inclusion and exclusion
- Webhooks: HTTP notifications on task completion
- Streaming events view for each Task in the Task Manager
- OAuth flow for easy testing

## The Architecture

The task streaming playground we're building includes:

- OAuth2 authentication with Parallel's identity provider
- A comprehensive task creation form supporting all processor types and output schemas
- Real-time task progress streaming with Server-Sent Events
- Task history management with persistent localStorage
- Auto-reconnection for resilient streaming connections
- Rich event visualization with progress indicators and final outputs

## Our technology stack

- Parallel Task API for task execution
- Parallel OAuth Provider for secure authentication
- Server-Sent Events[Server-Sent Events]($https://docs.parallel.ai/task-api/features/task-sse) for real-time streaming
- Cloudflare Workers[Cloudflare Workers]($https://workers.cloudflare.com/) for deployment and CORS proxying
- Pure HTML/JavaScript/CSS for maximum compatibility

## Why this architecture

### Stateless Streaming Design

The key insight behind this implementation is that you don't need to maintain any backend state during streaming. The Parallel Task API's SSE endpoint provides the complete current state every time you connect, including:

- All previous events that occurred before connecting
- The latest progress statistics
- Current task status and metadata
- Final outputs when tasks complete

This stateless design means your backend can be incredibly simple - just a CORS proxy. All the complexity lives in the well-tested Parallel infrastructure.

### OAuth2 with PKCE Security

For production-ready authentication, we implement the complete OAuth2 flow with PKCE (Proof Key for Code Exchange):

Dynamic Client Registration: Register OAuth client on-demand
PKCE Challenge: Generate cryptographically secure code challenge
Authorization Redirect: Send user to Parallel's OAuth server
Token Exchange: Securely exchange authorization code for access token
Persistent Storage: Store token in localStorage for session management

This provides enterprise-grade security without requiring pre-registration of OAuth clients.

## Implementation

### Real-Time Event Streaming

Server-Sent Events (SSE) provide the real-time updates for this Task manager that helps end-users understand the progress of the Parallel Task API. This is especially helpful for longer-running processors, like Pro and above. Unlike traditional polling approaches that repeatedly ask "are you done yet?", SSE creates a persistent connection that streams updates as they happen. However, implementing SSE correctly in the browser requires handling several complex challenges.

#### Why Manual SSE Implementation?

While browsers provide a built-in `EventSource` API for SSE, it has a critical limitation: **you cannot set custom headers**. Since Parallel's API requires authentication via the `x-api-key` header, we must implement SSE manually using the Fetch API and ReadableStreams.

SSE data arrives as a continuous stream of bytes, not discrete messages. Network packets can split messages in unpredictable ways:

Our implementation handles this with a streaming buffer pattern:

### Streaming buffer pattern

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  // Add new data to buffer
  buffer += decoder.decode(value, { stream: true });
  
  // Split into complete lines
  const lines = buffer.split('\n');
  buffer = lines.pop(); // Keep incomplete line in buffer
  
  // Process complete lines
  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = JSON.parse(line.substring(6));
      handleEvent(data);
    }
  }
}``` const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
 
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
 
  // Add new data to buffer
  buffer += decoder.decode(value, { stream: true });
  
  // Split into complete lines
  const lines = buffer.split('\n');
  buffer = lines.pop(); // Keep incomplete line in buffer
  
  // Process complete lines
  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = JSON.parse(line.substring(6));
      handleEvent(data);
    }
  }
}
```

This pattern ensures we never lose data or attempt to parse incomplete JSON, regardless of how network packets arrive.

#### Resilient Connection Management

Task execution can take anywhere from seconds to 30+ minutes. Network connections inevitably drop during long operations, so robust reconnection logic is essential:

### Auto-retry logic for active tasks

1
2
3
4
5
6
7
8
if (streamReconnectAttempts < MAX_RECONNECT_ATTEMPTS &&
    currentTaskRun && ['queued', 'running'].includes(currentTaskRun.status)) {
  streamReconnectAttempts++;
  setTimeout(() => {
    console.log(`Reconnecting attempt ${streamReconnectAttempts}...`);
    startStream();
  }, 2000);
}``` if (streamReconnectAttempts < MAX_RECONNECT_ATTEMPTS &&
    currentTaskRun && ['queued', 'running'].includes(currentTaskRun.status)) {
  streamReconnectAttempts++;
  setTimeout(() => {
    console.log(`Reconnecting attempt ${streamReconnectAttempts}...`);
    startStream();
  }, 2000);
}
```

**Key resilience features:**

- Status-based reconnection: Only retry for tasks that might still be generating events
- Exponential backoff: 2-second delays prevent overwhelming the server
- Attempt limiting: Prevents infinite retry loops on permanent failures
- Stateless recovery: Each reconnection gets the complete current state

#### Understanding the Event Stream Format

Parallel's SSE stream follows the standard Server-Sent Events specification. Each event is prefixed with `data: ` and terminated with double newlines. The implementation strips the prefix and parses the JSON payload:

### Standard SEE specification

1
2
3
4
5
6
7
8
9
10
11
for (const line of lines) {
  if (line.startsWith('data: ')) {
    try {
      const data = JSON.parse(line.substring(6)); // Remove 'data: ' prefix
      handleEvent(data);
    } catch (error) {
      console.error('Error parsing event data:', error, line);
    }
  }
}``` for (const line of lines) {
  if (line.startsWith('data: ')) {
    try {
      const data = JSON.parse(line.substring(6)); // Remove 'data: ' prefix
      handleEvent(data);
    } catch (error) {
      console.error('Error parsing event data:', error, line);
    }
  }
}
 
```

#### Event Type Taxonomy

The stream delivers several categories of events, each serving a different purpose:

- `task_run.state`: Core lifecycle events (queued → running → completed/failed)
- `task_run.progress_stats`: Quantitative metrics (sources found, pages read, tokens used)
- `task_run.progress_msg.*`: Qualitative updates with timestamped reasoning steps
- `task_run.progress_msg.reasoning`: AI thought process
- `task_run.progress_msg.search`: Search query generation
- `task_run.progress_msg.analysis`: Content analysis steps
- `error`: Exception conditions with detailed error messages

This rich event taxonomy enables granular UI updates - progress bars for stats, reasoning displays for AI thinking, and error handling for failures.

#### Event Visualization and UI

The frontend renders different event types with appropriate styling:

### Displaying the events

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
function handleEvent(data) {
  console.log(`event [${data.type}]`, data);
  
  let eventHtml = '';
  let eventClass = '';

  switch (data.type) {
    case 'task_run.state':
      eventClass = 'state';
      eventHtml = `
        <div class="event-type">TASK STATE</div>
        <div class="event-message">
          Status: <span class="status ${data.run.status}">${data.run.status}</span>
          ${data.run.error ? `<br><span class="error">Error: ${data.run.error.message}</span>` : ''}
        </div>
      `;

      // Display final output
      if (data.output) {
        const outputHtml = formatOutput(data.output);
        eventHtml += `<div class="output-section">
          <strong>Final Output:</strong>
          ${outputHtml}
        </div>`;
      }
      break;

    case 'task_run.progress_stats':
      eventClass = 'progress-stats';
      const stats = data.source_stats;
      eventHtml = `
        <div class="event-type">PROGRESS STATS</div>
        <div class="progress-stats">
          <div class="stat">Sources Considered: ${stats.num_sources_considered || 'N/A'}</div>
          <div class="stat">Sources Read: ${stats.num_sources_read || 'N/A'}</div>
        </div>
        ${stats.sources_read_sample ? `
          <div style="margin-top: 10px;">
            <strong>Sample Sources:</strong>
            <ul style="margin-top: 5px; margin-left: 20px;">
              ${stats.sources_read_sample.slice(0, 3).map(url => ``` function handleEvent(data) {
  console.log(`event [${data.type}]`, data);
  
  let eventHtml = '';
  let eventClass = '';
 
  switch (data.type) {
    case 'task_run.state':
      eventClass = 'state';
      eventHtml = `
        <div class="event-type">TASK STATE</div>
        <div class="event-message">
          Status: <span class="status ${data.run.status}">${data.run.status}</span>
          ${data.run.error ? `<br><span class="error">Error: ${data.run.error.message}</span>` : ''}
        </div>
      `;
 
      // Display final output
      if (data.output) {
        const outputHtml = formatOutput(data.output);
        eventHtml += `<div class="output-section">
          <strong>Final Output:</strong>
          ${outputHtml}
        </div>`;
      }
      break;
 
    case 'task_run.progress_stats':
      eventClass = 'progress-stats';
      const stats = data.source_stats;
      eventHtml = `
        <div class="event-type">PROGRESS STATS</div>
        <div class="progress-stats">
          <div class="stat">Sources Considered: ${stats.num_sources_considered || 'N/A'}</div>
          <div class="stat">Sources Read: ${stats.num_sources_read || 'N/A'}</div>
        </div>
        ${stats.sources_read_sample ? `
          <div style="margin-top: 10px;">
            <strong>Sample Sources:</strong>
            <ul style="margin-top: 5px; margin-left: 20px;">
              ${stats.sources_read_sample.slice(0, 3).map(url => 
```

### CORS Proxy Worker

A simple worker solves a fundamental web security challenge. Modern browsers implement the Same-Origin Policy, which prevents JavaScript running on `yourdomain.com` from making direct API calls to `api.parallel.ai`. This security feature protects users from malicious websites, but it also blocks legitimate applications from accessing external APIs.

Traditional solutions involve building a full backend API server that handles authentication, request validation, and response transformation. Instead, we use a much simpler proxy pattern that acts as a transparent middleman:

- Path Rewriting: `/tasks-sse/api/v1/tasks/runs` becomes `https://api.parallel.ai/v1/tasks/runs`
- Header Preservation: All original headers (including `x-api-key`) are forwarded
- CORS Headers: Added to all responses to enable browser access
- Static Serving: The same worker serves the HTML frontend

### CORS Proxy worker

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
// @ts-ignore
import indexHtml from "./index.html";

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);

    // Serve the frontend
    if (url.pathname === "/tasks-sse/") {
      return new Response(indexHtml, {
        headers: { "Content-Type": "text/html;charset=utf8" },
      });
    }

    // Proxy Parallel API requests
    if (url.pathname.startsWith("/tasks-sse/api/")) {
      // Remove /api prefix and forward to api.parallel.ai
      const targetPath = url.pathname.replace("/tasks-sse/api", "");
      const targetUrl = `https://api.parallel.ai${targetPath}${url.search}`;

      // Clone the request but change the URL
      const modifiedRequest = new Request(targetUrl, {
        method: request.method,
        headers: request.headers,
        body: request.body,
      });

      // Forward the request
      const response = await fetch(modifiedRequest);

      // Clone the response to modify headers
      const modifiedResponse = new Response(response.body, {
        status: response.status,
        statusText: response.statusText,
        headers: {
          ...Object.fromEntries(response.headers),
          // Add CORS headers
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",
          "Access-Control-Allow-Headers": "*",
        },
      });

      return modifiedResponse;
    }

    // Handle preflight requests
    if (request.method === "OPTIONS") {
      return new Response(null, {
        headers: {
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",
          "Access-Control-Allow-Headers": "*",
        },
      });
    }

    return new Response(null, {
      status: 302,
      headers: { Location: "/tasks-sse/" },
    });
  },
};``` // @ts-ignore
import indexHtml from "./index.html";
 
export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
 
    // Serve the frontend
    if (url.pathname === "/tasks-sse/") {
      return new Response(indexHtml, {
        headers: { "Content-Type": "text/html;charset=utf8" },
      });
    }
 
    // Proxy Parallel API requests
    if (url.pathname.startsWith("/tasks-sse/api/")) {
      // Remove /api prefix and forward to api.parallel.ai
      const targetPath = url.pathname.replace("/tasks-sse/api", "");
      const targetUrl = `https://api.parallel.ai${targetPath}${url.search}`;
 
      // Clone the request but change the URL
      const modifiedRequest = new Request(targetUrl, {
        method: request.method,
        headers: request.headers,
        body: request.body,
      });
 
      // Forward the request
      const response = await fetch(modifiedRequest);
 
      // Clone the response to modify headers
      const modifiedResponse = new Response(response.body, {
        status: response.status,
        statusText: response.statusText,
        headers: {
          ...Object.fromEntries(response.headers),
          // Add CORS headers
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",
          "Access-Control-Allow-Headers": "*",
        },
      });
 
      return modifiedResponse;
    }
 
    // Handle preflight requests
    if (request.method === "OPTIONS") {
      return new Response(null, {
        headers: {
          "Access-Control-Allow-Origin": "*",
          "Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",
          "Access-Control-Allow-Headers": "*",
        },
      });
    }
 
    return new Response(null, {
      status: 302,
      headers: { Location: "/tasks-sse/" },
    });
  },
};
```

### OAuth Authentication

The OAuth flow implementation handles dynamic client registration, as well as the complete PKCE security protocol:

### OAuth implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
async function startOAuth() {
  try {
    // Register client dynamically
    const reg = await fetch("https://platform.parallel.ai/getKeys/register", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        redirect_uris: [window.location.origin + window.location.pathname]
      }),
    });

    const { client_id } = await reg.json();

    // Generate PKCE challenge
    const cv = btoa(crypto.getRandomValues(new Uint8Array(32))).replace(
      /[+/=]/g, (m) => ({ "+": "-", "/": "_", "=": "" }[m])
    );
    localStorage.setItem('code_verifier', cv);

    const cc = btoa(
      String.fromCharCode(
        ...new Uint8Array(
          await crypto.subtle.digest(
            "SHA-256", 
            new TextEncoder().encode(cv)
          )
        )
      )
    ).replace(/[+/=]/g, (m) => ({ "+": "-", "/": "_", "=": "" }[m]));

    // Redirect to OAuth server
    const url = new URL("https://platform.parallel.ai/getKeys/authorize");
    Object.entries({
      client_id,
      redirect_uri: window.location.origin + window.location.pathname,
      response_type: "code",
      scope: "api",
      code_challenge: cc,
      code_challenge_method: "S256",
      state: Math.random().toString(36).substring(7)
    }).forEach(([k, v]) => url.searchParams.set(k, v));

    window.location.href = url;
  } catch (error) {
    alert('OAuth error: ' + error.message);
  }
}``` async function startOAuth() {
  try {
    // Register client dynamically
    const reg = await fetch("https://platform.parallel.ai/getKeys/register", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        redirect_uris: [window.location.origin + window.location.pathname]
      }),
    });
 
    const { client_id } = await reg.json();
 
    // Generate PKCE challenge
    const cv = btoa(crypto.getRandomValues(new Uint8Array(32))).replace(
      /[+/=]/g, (m) => ({ "+": "-", "/": "_", "=": "" }[m])
    );
    localStorage.setItem('code_verifier', cv);
 
    const cc = btoa(
      String.fromCharCode(
        ...new Uint8Array(
          await crypto.subtle.digest(
            "SHA-256", 
            new TextEncoder().encode(cv)
          )
        )
      )
    ).replace(/[+/=]/g, (m) => ({ "+": "-", "/": "_", "=": "" }[m]));
 
    // Redirect to OAuth server
    const url = new URL("https://platform.parallel.ai/getKeys/authorize");
    Object.entries({
      client_id,
      redirect_uri: window.location.origin + window.location.pathname,
      response_type: "code",
      scope: "api",
      code_challenge: cc,
      code_challenge_method: "S256",
      state: Math.random().toString(36).substring(7)
    }).forEach(([k, v]) => url.searchParams.set(k, v));
 
    window.location.href = url;
  } catch (error) {
    alert('OAuth error: ' + error.message);
  }
}
```

## Resources

- App Source Code[App Source Code]($https://github.com/parallel-web/parallel-cookbook/tree/main/typescript-recipes/parallel-tasks-sse)
- Live App Example[Live App Example]($https://oss.parallel.ai/tasks-sse/)
- Parallel SSE Documentation[Parallel SSE Documentation]($https://docs.parallel.ai/task-api/features/task-sse)

By Parallel

Tags:Cookbook

Reading time: 6 min

Introducing Parallel Deep Research reports

- [Introducing Basis with Calibrated Confidences ](https://parallel.ai/blog/introducing-basis-with-calibrated-confidences)

Tags:Product Release

Reading time: 4 min

The Parallel Task API is a state-of-the-art system for automated web research that delivers the highest accuracy at every price point.

- [Introducing the Parallel Task API](https://parallel.ai/blog/parallel-task-api)

Tags:Product Release,Benchmarks

Reading time: 4 min

# Building a real-time streaming task manager with Parallel

## Key Features

## The Architecture

## Our technology stack

## Why this architecture

### Stateless Streaming Design

### OAuth2 with PKCE Security

Architecture diagram

## Implementation

### Real-Time Event Streaming

#### Why Manual SSE Implementation?

#### Resilient Connection Management

#### Understanding the Event Stream Format

#### Event Type Taxonomy

#### Event Visualization and UI

### CORS Proxy Worker

### OAuth Authentication

## Resources

## Related Posts29

- [How Starbridge powers public sector GTM with state-of-the-art web research](https://parallel.ai/blog/case-study-starbridge)

- [Building a market research platform with Parallel Deep Research](https://parallel.ai/blog/cookbook-market-research-platform-with-parallel)

- [How Lindy brings state-of-the-art web research to automation flows](https://parallel.ai/blog/case-study-lindy)

- [Introducing the Parallel Task MCP Server](https://parallel.ai/blog/parallel-task-mcp-server)

- [Introducing the Core2x Processor for improved compute control on the Task API](https://parallel.ai/blog/core2x-processor)

- [How Day AI merges private and public data for business intelligence](https://parallel.ai/blog/case-study-day-ai)

- [Full Basis framework for all Task API Processors](https://parallel.ai/blog/full-basis-framework-for-task-api)

- [How Gumloop built a new AI automation framework with web intelligence as a core node](https://parallel.ai/blog/case-study-gumloop)

- [Introducing the TypeScript SDK](https://parallel.ai/blog/typescript-sdk)

- [Building a serverless competitive intelligence platform with MCP + Task API](https://parallel.ai/blog/cookbook-competitor-research-with-reddit-mcp)

- [Introducing Parallel Deep Research reports](https://parallel.ai/blog/deep-research-reports)

- [A new pareto-frontier for Deep Research price-performance](https://parallel.ai/blog/deep-research-benchmarks)

- [Building a Full-Stack Search Agent with Parallel and Cerebras](https://parallel.ai/blog/cookbook-search-agent)

- [Webhooks for the Parallel Task API](https://parallel.ai/blog/webhooks)

- [Introducing Parallel: Web Search Infrastructure for AIs ](https://parallel.ai/blog/introducing-parallel)

- [Introducing SSE for Task Runs](https://parallel.ai/blog/sse-for-tasks)

- [A new line of advanced processors: Ultra2x, Ultra4x, and Ultra8x ](https://parallel.ai/blog/new-advanced-processors)

- [Introducing Auto Mode for the Parallel Task API](https://parallel.ai/blog/task-api-auto-mode)

- [A state-of-the-art search API purpose-built for agents](https://parallel.ai/blog/search-api-benchmark)

- [Parallel Search MCP Server in Devin](https://parallel.ai/blog/parallel-search-mcp-in-devin)

- [Introducing Tool Calling via MCP Servers](https://parallel.ai/blog/mcp-tool-calling)

- [Introducing the Parallel Search MCP Server ](https://parallel.ai/blog/search-mcp-server)

- [Introducing Source Policy](https://parallel.ai/blog/source-policy)

- [The Parallel Task Group API](https://parallel.ai/blog/task-group-api)

- [State of the Art Deep Research APIs](https://parallel.ai/blog/deep-research)

- [Introducing the Parallel Search API ](https://parallel.ai/blog/parallel-search-api)

- [Introducing the Parallel Chat API ](https://parallel.ai/blog/chat-api)

- [Introducing Basis with Calibrated Confidences ](https://parallel.ai/blog/introducing-basis-with-calibrated-confidences)

- [Introducing the Parallel Task API](https://parallel.ai/blog/parallel-task-api)

Info