v0.1.0-beta

OpenAPI 3.1.0

Live OnPage API Content Parsing

Felo API Support

MIT

Server

https://openapi.felo.ai

Client Libraries

Shell Curl

DataForSEO

SEO, search-engine results, keyword research, backlinks and AI visibility data.

DataForSEO Operations

post/v1/beta/dataforseo/on_page/content_parsing/live

Live OnPage API Content Parsing

Parses any url into structured content immediately, with no crawl needed - the only tool in this family that works standalone. browser_preset, browser_screen_width, browser_screen_height and custom_user_agent control the render, disable_cookie_popup removes consent overlays, store_raw_html keeps the source. Wrapped in DataForSEO's envelope: data in tasks[0].result, outcome in tasks[0].status_code - a rejected request still returns HTTP 200. For pages already inside a crawl use post_dataforseo_on_page_content_parsing, which is free.

Body

required

application/json

Type: array object[]
- url
  Type: string
  required
  
  URL of the content to parse required field URL of the page to parse example: https://www.fujielectric.com/
- accept_language
  Type: string
  
  language header for accessing the website optional field all locale formats are supported (xx, xx-XX, xxx-XX, etc.) Note: if you do not specify this parameter, some websites may deny access; in this case, pages will be returned with the "type":"broken in the response array
- browser_preset
  Type: string
  
  preset for browser screen parameters optional field if you use this field, you don’t need to indicate browser_screen_width, browser_screen_height, browser_screen_scale_factor possible values: desktop, mobile, tablet desktop preset will apply the following values: browser_screen_width: 1920 browser_screen_height: 1080 browser_screen_scale_factor: 1 mobile preset will apply the following values: browser_screen_width: 390 browser_screen_height: 844 browser_screen_scale_factor: 3 tablet preset will apply the following values: browser_screen_width: 1024 browser_screen_height: 1366 browser_screen_scale_factor: 2 Note: to use this parameter, set enable_javascript or enable_browser_rendering to true
- browser_screen_height
  Type: integer
  
  browser screen height optional field you can set a custom browser screen height to perform audit for a particular device; if you use this field, you don’t need to indicate browser_preset as it will be ignored; Note: to use this parameter, set enable_javascript or enable_browser_rendering to true minimum value, in pixels: 240 maximum value, in pixels: 9999
- browser_screen_scale_factor
  Type: number
  
  browser screen scale factor optional field you can set a custom browser screen resolution ratio to perform audit for a particular device; if you use this field, you don’t need to indicate browser_preset as it will be ignored; Note: to use this parameter, set enable_javascript or enable_browser_rendering to true minimum value: 0.5 maximum value: 3
- browser_screen_width
  Type: integer
  
  browser screen width optional field you can set a custom browser screen width to perform audit for a particular device; if you use this field, you don’t need to indicate browser_preset as it will be ignored; Note: to use this parameter, set enable_javascript or enable_browser_rendering to true minimum value, in pixels: 240 maximum value, in pixels: 9999
- custom_user_agent
  Type: string
  
  custom user agent optional field custom user agent for crawling a website example: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 default value: Mozilla/5.0 (compatible; RSiteAuditor)
- disable_cookie_popup
  Type: boolean
  
  disable the cookie popup optional field set to true if you want to disable the popup requesting cookie consent from the user; default value: false
- enable_browser_rendering
  Type: boolean
  
  emulate browser rendering to measure Core Web Vitals optional field by using this parameter you will be able to emulate a browser when loading a web page; enable_browser_rendering loads styles, images, fonts, animations, videos, and other resources on a page; default value: false set to true to obtain Core Web Vitals (FID, CLS, LCP) metrics in the response; if you use this field, enable_javascript, and load_resources parameters must be set to true Note: if you use this parameter, additional charges will apply; learn more about the cost of tasks with this parameter in our help article; the cost can be calculated on the Pricing Page
- enable_javascript
  Type: boolean
  
  load javascript on a page optional field set to true if you want to load the scripts available on a page default value: false Note: if you use this parameter, additional charges will apply; learn more about the cost of tasks with this parameter in our help article; the cost can be calculated on the Pricing Page
- enable_xhr
  Type: boolean
  
  enable XMLHttpRequest on a page optional field set to true if you want our crawler to request data from a web server using the XMLHttpRequest object default value: false if you use this field, enable_javascript must be set to true;
- ip_pool_for_scan
  Type: string
  
  proxy pool optional field you can choose a location of the proxy pool that will be used to obtain the requested data; the parameter can be used if page content is inaccessible in one of the locations, resulting in occasional site_unreachable errors possible values: us, de
- markdown_view
  Type: boolean
  
  return page content as markdown optional field if set to true, the markdown-formatted content of the page will be returned in the page_as_markdown field of the response; default value: false
- store_raw_html
  Type: boolean
  
  store HTML of a crawled page optional field set to true if you want to get the HTML of the page using the OnPage Raw HTML endpoint default value: false
- switch_pool
  Type: boolean
  
  switch proxy pool optional field if true, additional proxy pools will be used to obtain the requested data; the parameter can be used if a multitude of tasks is set simultaneously, resulting in occasional rate-limit and/or site_unreachable errors

Responses

200
Type: object

Successful response
cost
Type: number

total tasks cost, USD

status_code
Type: integer

general status code you can find the full list of the response codes here Note: we strongly recommend designing a necessary system for handling related exceptional or error conditions

status_message
Type: string

general informational message you can find the full list of general informational messages here

tasks
Type: array string[]

array of tasks

tasks_count
Type: integer

the number of tasks in the tasks array

tasks_error
Type: integer

the number of tasks in the tasks array returned with an error

tasks.cost
Type: number

cost of the task, USD

tasks.data
Type: object

contains the same parameters that you specified in the POST request

tasks.id
Type: string

task identifier unique task identifier in our system in the UUID format

tasks.path
Type: array string[]

URL path

tasks.result
Type: array string[]

array of results

tasks.result_count
Type: integer

number of elements in the result array

tasks.result.crawl_progress
Type: string

status of the crawling session possible values: in_progress, finished

tasks.result.crawl_status
Type: object

details of the crawling session

tasks.result.items
Type: array string[]

items array

tasks.result.items_count
Type: integer

number of items in the results array

tasks.result.items.fetch_time
Type: string

date and time when the content was fetched in the UTC format: “yyyy-mm-dd hh-mm-ss +00:00” example: "2022-11-01 10:02:52 +00:00"

tasks.result.items.page_as_markdown
Type: string

page content in the markdown format page content in the text-to-HTML markdown format specify markdown_view as true in the request to return the value

tasks.result.items.page_content
Type: object

parsed content of the page

tasks.result.items.page_content.comments
Type: array string[]

array of comments displayed on the page contains objects with information on comments related to displayed products

tasks.result.items.page_content.comments.author
Type: string

author of the comment

tasks.result.items.page_content.comments.max_rating_value
Type: integer

maximum value for the rating

tasks.result.items.page_content.comments.name
Type: string

rating name Note: this field is not used in this particular object, and its value is always null

tasks.result.items.page_content.comments.primary_content
Type: array string[]

primary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.comments.primary_content.text
Type: string

text of the comment

tasks.result.items.page_content.comments.primary_content.url
Type: string

displayed in case the text is a link anchor

tasks.result.items.page_content.comments.primary_content.urls
Type: array string[]

contains other URLs and anchors found in the content element

tasks.result.items.page_content.comments.publish_date
Type: string

date when the comment was published

tasks.result.items.page_content.comments.rating
Type: object

product’s rating contains information about the rating a customer has given to the product

tasks.result.items.page_content.comments.rating_count
Type: integer

the amount of feedback Note: this field is not used in this particular object, and its value is always null

tasks.result.items.page_content.comments.rating_value
Type: integer

the value of the rating

tasks.result.items.page_content.comments.relative rating
Type: number

relative rating can take values from 0 to 1

tasks.result.items.page_content.comments.title
Type: string

title of the customer’s comment

tasks.result.items.page_content.contacts
Type: object

contact information contains contact information displayed on the page

tasks.result.items.page_content.contacts.emails
Type: array string[]

array of emails

tasks.result.items.page_content.contacts.telephones
Type: array string[]

array of telephone numbers

tasks.result.items.page_content.footer
Type: object

parsed content of the footer

tasks.result.items.page_content.header
Type: object

parsed content of the header

tasks.result.items.page_content.main_topic
Type: array string[]

main topic on the page you can find more information about topic priority calculation in this help center article

tasks.result.items.page_content.main_topic.author
Type: string

content author name

tasks.result.items.page_content.main_topic.h_title
Type: string

meta title

tasks.result.items.page_content.main_topic.language
Type: string

content language

tasks.result.items.page_content.main_topic.level
Type: string

HTML level

tasks.result.items.page_content.main_topic.main_title
Type: string

main title of the block

tasks.result.items.page_content.main_topic.primary_content
Type: array string[]

primary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.main_topic.secondary_content
Type: array string[]

secondary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.main_topic.text
Type: string

content text

tasks.result.items.page_content.main_topic.url
Type: string

page URL displayed in case the text is a link anchor

tasks.result.items.page_content.main_topic.urls
Type: array string[]

contains other URLs and anchors found in the content element

tasks.result.items.page_content.main_topic.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.main_topic.urls.url
Type: string

other URL found in the content element

tasks.result.items.page_content.offers
Type: array string[]

array of products displayed on the page contains objects with information on products displayed on the page

tasks.result.items.page_content.offers.name
Type: string

name of the product

tasks.result.items.page_content.offers.price
Type: integer

price of the product

tasks.result.items.page_content.offers.price_currency
Type: string

price currency

tasks.result.items.page_content.offers.price_valid_until
Type: integer

displays the date and time until which the price is valid in the UTC format: “yyyy-mm-dd hh-mm-ss +00:00” example: "2022-11-01 10:02:52 +00:00"

tasks.result.items.page_content.primary_content
Type: array string[]

primary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.primary_content.text
Type: string

content text

tasks.result.items.page_content.primary_content.url
Type: string

page URL displayed in case the text is a link anchor

tasks.result.items.page_content.primary_content.urls
Type: array string[]

contains other URLs and anchors found in the content element

tasks.result.items.page_content.primary_content.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.primary_content.urls.url
Type: string

other URL found in the content element

tasks.result.items.page_content.ratings
Type: array string[]

contains objects with rating information for the products displayed on the page

tasks.result.items.page_content.ratings.max_rating_value
Type: integer

maximum value for the rating

tasks.result.items.page_content.ratings.name
Type: string

rating name Note: this field is not used in this particular object, and its value is always set to null

tasks.result.items.page_content.ratings.rating_count
Type: integer

the amount of feedback

tasks.result.items.page_content.ratings.rating_value
Type: integer

the value of the rating

tasks.result.items.page_content.ratings.relative_rating
Type: number

relative rating can take values from 0 to 1

tasks.result.items.page_content.secondary_content
Type: array string[]

secondary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.secondary_content.text
Type: string

content text

tasks.result.items.page_content.secondary_content.url
Type: string

page URL displayed in case the text is a link anchor

tasks.result.items.page_content.secondary_content.urls
Type: array string[]

contains other URLs and anchors found in the content element

tasks.result.items.page_content.secondary_content.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.secondary_content.urls.url
Type: string

other URL found in the content element

tasks.result.items.page_content.secondary_topic
Type: array string[]

secondary topic on the page you can find more information about topic priority calculation in this help center article

tasks.result.items.page_content.secondary_topic.author
Type: string

content author name

tasks.result.items.page_content.secondary_topic.h_title
Type: string

meta title

tasks.result.items.page_content.secondary_topic.language
Type: string

content language

tasks.result.items.page_content.secondary_topic.level
Type: string

HTML level

tasks.result.items.page_content.secondary_topic.main_title
Type: string

main title of the block

tasks.result.items.page_content.secondary_topic.primary_content
Type: array string[]

primary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.secondary_topic.secondary_content
Type: array string[]

secondary content on the page you can find more information about content priority calculation in this help center article

tasks.result.items.page_content.secondary_topic.text
Type: string

content text

tasks.result.items.page_content.secondary_topic.url
Type: string

page URL displayed in case the text is a link anchor

tasks.result.items.page_content.secondary_topic.urls
Type: array string[]

contains other URLs and anchors found in the content element

tasks.result.items.page_content.secondary_topic.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.secondary_topic.urls.url
Type: string

other URL found in the content element

tasks.result.items.page_content.table_content
Type: array string[]

content of the table on the page

tasks.result.items.page_content.table_content.body
Type: array string[]

content of the body of the table

tasks.result.items.page_content.table_content.body.row_cells
Type: array string[]

content of the row cells of the header

tasks.result.items.page_content.table_content.body.row_cells.is_header
Type: boolean

indicates if the text belongs to the header

tasks.result.items.page_content.table_content.body.row_cells.text
Type: string

text in the row cell

tasks.result.items.page_content.table_content.body.row_cells.urls
Type: array string[]

contains other URLs and anchors found in the cell

tasks.result.items.page_content.table_content.body.row_cells.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.table_content.body.row_cells.urls.url
Type: string

URL found in the cell

tasks.result.items.page_content.table_content.footer
Type: array string[]

content of the footer of the table

tasks.result.items.page_content.table_content.footer.row_cells
Type: array string[]

content of the row cells of the header

tasks.result.items.page_content.table_content.footer.row_cells.is_header
Type: boolean

indicates if the text belongs to the header

tasks.result.items.page_content.table_content.footer.row_cells.text
Type: string

text in the row cell

tasks.result.items.page_content.table_content.footer.row_cells.urls
Type: array string[]

contains other URLs and anchors found in the cell

tasks.result.items.page_content.table_content.footer.row_cells.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.table_content.footer.row_cells.urls.url
Type: string

URL found in the cell

tasks.result.items.page_content.table_content.header
Type: array string[]

content of the header of the table

tasks.result.items.page_content.table_content.header.row_cells
Type: array string[]

content of the row cells of the header

tasks.result.items.page_content.table_content.header.row_cells.is_header
Type: boolean

indicates if the text belongs to the header

tasks.result.items.page_content.table_content.header.row_cells.text
Type: string

text in the row cell

tasks.result.items.page_content.table_content.header.row_cells.urls
Type: array string[]

contains other URLs and anchors found in the cell

tasks.result.items.page_content.table_content.header.row_cells.urls.anchor_text
Type: string

text of the URL’s anchor

tasks.result.items.page_content.table_content.header.row_cells.urls.url
Type: string

URL found in the cell

tasks.result.items.status_code
Type: integer

status code of the page

tasks.result.items.type
Type: string

type of the returned item = ‘сontent_parsing_element’

tasks.status_code
Type: integer

status code of the task generated by DataForSEO; can be within the following range: 10000-60000 you can find the full list of the response codes here

tasks.status_message
Type: string

informational message of the task you can find the full list of general informational messages here

tasks.time
Type: string

execution time, seconds

time
Type: string

execution time, seconds

version
Type: string

the current version of the API
application/json
400

Bad request
401

Unauthorized
402

The request cannot proceed because a billing requirement is not met.
403

The account is not permitted to perform this operation.
429

Rate limit exceeded
500

Internal server error
502

The service could not complete the request.
503

The API or billing service is temporarily unavailable.
504

The service timed out while processing the request.
default

The operation failed. Keep the response request ID when contacting Felo support.

post/v1/beta/dataforseo/on_page/content_parsing/live

curl https://openapi.felo.ai/v1/beta/dataforseo/on_page/content_parsing/live \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer YOUR_SECRET_TOKEN' \
  --data '[
  {
    "url": "",
    "custom_user_agent": "",
    "browser_preset": "",
    "browser_screen_width": 1,
    "browser_screen_height": 1,
    "browser_screen_scale_factor": 1,
    "store_raw_html": true,
    "disable_cookie_popup": true,
    "accept_language": "",
    "enable_javascript": true,
    "enable_browser_rendering": true,
    "enable_xhr": true,
    "switch_pool": true,
    "ip_pool_for_scan": "",
    "markdown_view": true
  }
]'

Show Schema

{
  "version": "string",
  "status_code": 1,
  "status_message": "string",
  "time": "string",
  "cost": 1,
  "tasks_count": 1,
  "tasks_error": 1,
  "tasks": [
    "string"
  ],
  "tasks.id": "string",
  "tasks.status_code": 1,
  "tasks.status_message": "string",
  "tasks.time": "string",
  "tasks.cost": 1,
  "tasks.result_count": 1,
  "tasks.path": [
    "string"
  ],
  "tasks.data": {},
  "tasks.result": [
    "string"
  ],
  "tasks.result.crawl_progress": "string",
  "tasks.result.crawl_status": {},
  "tasks.result.items_count": 1,
  "tasks.result.items": [
    "string"
  ],
  "tasks.result.items.type": "string",
  "tasks.result.items.fetch_time": "string",
  "tasks.result.items.status_code": 1,
  "tasks.result.items.page_content": {},
  "tasks.result.items.page_content.header": {},
  "tasks.result.items.page_content.primary_content": [
    "string"
  ],
  "tasks.result.items.page_content.primary_content.text": "string",
  "tasks.result.items.page_content.primary_content.url": "string",
  "tasks.result.items.page_content.primary_content.urls": [
    "string"
  ],
  "tasks.result.items.page_content.primary_content.urls.url": "string",
  "tasks.result.items.page_content.primary_content.urls.anchor_text": "string",
  "tasks.result.items.page_content.secondary_content": [
    "string"
  ],
  "tasks.result.items.page_content.secondary_content.text": "string",
  "tasks.result.items.page_content.secondary_content.url": "string",
  "tasks.result.items.page_content.secondary_content.urls": [
    "string"
  ],
  "tasks.result.items.page_content.secondary_content.urls.url": "string",
  "tasks.result.items.page_content.secondary_content.urls.anchor_text": "string",
  "tasks.result.items.page_content.table_content": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.header": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.header.row_cells": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.header.row_cells.text": "string",
  "tasks.result.items.page_content.table_content.header.row_cells.urls": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.header.row_cells.urls.url": "string",
  "tasks.result.items.page_content.table_content.header.row_cells.urls.anchor_text": "string",
  "tasks.result.items.page_content.table_content.header.row_cells.is_header": true,
  "tasks.result.items.page_content.table_content.body": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.body.row_cells": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.body.row_cells.text": "string",
  "tasks.result.items.page_content.table_content.body.row_cells.urls": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.body.row_cells.urls.url": "string",
  "tasks.result.items.page_content.table_content.body.row_cells.urls.anchor_text": "string",
  "tasks.result.items.page_content.table_content.body.row_cells.is_header": true,
  "tasks.result.items.page_content.table_content.footer": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.footer.row_cells": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.footer.row_cells.text": "string",
  "tasks.result.items.page_content.table_content.footer.row_cells.urls": [
    "string"
  ],
  "tasks.result.items.page_content.table_content.footer.row_cells.urls.url": "string",
  "tasks.result.items.page_content.table_content.footer.row_cells.urls.anchor_text": "string",
  "tasks.result.items.page_content.table_content.footer.row_cells.is_header": true,
  "tasks.result.items.page_content.footer": {},
  "tasks.result.items.page_content.main_topic": [
    "string"
  ],
  "tasks.result.items.page_content.main_topic.h_title": "string",
  "tasks.result.items.page_content.main_topic.main_title": "string",
  "tasks.result.items.page_content.main_topic.author": "string",
  "tasks.result.items.page_content.main_topic.language": "string",
  "tasks.result.items.page_content.main_topic.level": "string",
  "tasks.result.items.page_content.main_topic.primary_content": [
    "string"
  ],
  "tasks.result.items.page_content.main_topic.text": "string",
  "tasks.result.items.page_content.main_topic.url": "string",
  "tasks.result.items.page_content.main_topic.urls": [
    "string"
  ],
  "tasks.result.items.page_content.main_topic.urls.url": "string",
  "tasks.result.items.page_content.main_topic.urls.anchor_text": "string",
  "tasks.result.items.page_content.main_topic.secondary_content": [
    "string"
  ],
  "tasks.result.items.page_content.secondary_topic": [
    "string"
  ],
  "tasks.result.items.page_content.secondary_topic.h_title": "string",
  "tasks.result.items.page_content.secondary_topic.main_title": "string",
  "tasks.result.items.page_content.secondary_topic.author": "string",
  "tasks.result.items.page_content.secondary_topic.language": "string",
  "tasks.result.items.page_content.secondary_topic.level": "string",
  "tasks.result.items.page_content.secondary_topic.primary_content": [
    "string"
  ],
  "tasks.result.items.page_content.secondary_topic.text": "string",
  "tasks.result.items.page_content.secondary_topic.url": "string",
  "tasks.result.items.page_content.secondary_topic.urls": [
    "string"
  ],
  "tasks.result.items.page_content.secondary_topic.urls.url": "string",
  "tasks.result.items.page_content.secondary_topic.urls.anchor_text": "string",
  "tasks.result.items.page_content.secondary_topic.secondary_content": [
    "string"
  ],
  "tasks.result.items.page_content.ratings": [
    "string"
  ],
  "tasks.result.items.page_content.ratings.name": "string",
  "tasks.result.items.page_content.ratings.rating_value": 1,
  "tasks.result.items.page_content.ratings.max_rating_value": 1,
  "tasks.result.items.page_content.ratings.rating_count": 1,
  "tasks.result.items.page_content.ratings.relative_rating": 1,
  "tasks.result.items.page_content.offers": [
    "string"
  ],
  "tasks.result.items.page_content.offers.name": "string",
  "tasks.result.items.page_content.offers.price": 1,
  "tasks.result.items.page_content.offers.price_currency": "string",
  "tasks.result.items.page_content.offers.price_valid_until": 1,
  "tasks.result.items.page_content.comments": [
    "string"
  ],
  "tasks.result.items.page_content.comments.rating": {},
  "tasks.result.items.page_content.comments.name": "string",
  "tasks.result.items.page_content.comments.rating_value": 1,
  "tasks.result.items.page_content.comments.max_rating_value": 1,
  "tasks.result.items.page_content.comments.rating_count": 1,
  "tasks.result.items.page_content.comments.relative rating": 1,
  "tasks.result.items.page_content.comments.title": "string",
  "tasks.result.items.page_content.comments.publish_date": "string",
  "tasks.result.items.page_content.comments.author": "string",
  "tasks.result.items.page_content.comments.primary_content": [
    "string"
  ],
  "tasks.result.items.page_content.comments.primary_content.text": "string",
  "tasks.result.items.page_content.comments.primary_content.url": "string",
  "tasks.result.items.page_content.comments.primary_content.urls": [
    "string"
  ],
  "tasks.result.items.page_content.contacts": {},
  "tasks.result.items.page_content.contacts.telephones": [
    "string"
  ],
  "tasks.result.items.page_content.contacts.emails": [
    "string"
  ],
  "tasks.result.items.page_as_markdown": "string"
}

Successful response

Live OnPage API Content Parsing

AuthenticationRequired

DataForSEO

Live OnPage API Content Parsing