Web URLs¶
Fetches content from web pages connected via the Carbon Web Scraper integration and returns them as a single formatted string. Each selected file (or chunk) is wrapped in a lightweight file tag so downstream nodes can parse or prompt with clear boundaries.
Usage¶
Use this node to bring web page content you've connected through Carbon's Web Scraper into your workflow. Typically, you first connect URLs using the Carbon connection modal, pick the files with the file picker, optionally provide a query to narrow results, and then pass the combined output to processing or LLM nodes.
Inputs¶
| Field | Required | Type | Description | Example |
|---|---|---|---|---|
| file_ids | True | CARBON_FILE_IDS | Identifiers of the Carbon files (web pages or scraped chunks) to fetch. This should be supplied by the Carbon file picker for the Web Scraper integration. | [101, 202, 303] |
| query | False | STRING | Optional filter to narrow the selection of content returned from the chosen files (for example, keywords or a brief phrase). | product pricing policy |
Outputs¶
| Field | Type | Description | Example |
|---|---|---|---|
| Output | STRING | A single concatenated string containing all fetched file contents, where each file is wrapped as: |
Important Notes¶
- Integration scope: This node specifically targets the Carbon Web Scraper integration; ensure you have connected web URLs via Carbon before use.
- File selection: Use the Carbon file picker to populate file_ids. Manually crafting file_ids must result in a valid JSON array string.
- Output format: Results are combined into one STRING with
tags; some IDs may include a -chunk:index suffix when content is chunked. - Filtering: The optional query input helps narrow returned content but does not replace selecting the correct files/URLs.
- Permissions: Access is restricted to the authenticated Salt user; content fetches respect your Carbon connection permissions.
Troubleshooting¶
- Invalid file_ids (JSON error): Ensure file_ids is a valid JSON array string (e.g., "[101, 202]") and that it comes from the Carbon file picker.
- No content returned: Verify the selected URLs are connected in Carbon, you have permission to access them, and the query isn't overly restrictive.
- Wrong integration: If you don't see your web URLs in the picker, confirm you're using the Web Scraper integration and that the connection is active.
- Downstream length limits: If subsequent nodes fail due to large input, reduce the number of selected files or apply a more specific query.