Data API / DataForSEO / On Page

OnPage API Non-indexable Pages

The crawled pages search engines will not index, each with a reason and url. Reads a finished crawl, so it needs the id from post_dataforseo_on_page_submit and returns crawl_progress plus a crawl_status of max_crawl_pages, pages_in_queue and pages_crawled - check those before trusting a small result, because a crawl still running simply has less to report. The reason field is the whole value here - noindex, canonical elsewhere, robots-blocked are very different problems with the same symptom. 💰 Free upstream: querying a finished crawl costs nothing, only the crawl itself does.

Параметры

Параметров пути, запроса и заголовков нет.

Тело запроса

idstringОбязательный
ID of the task required field you can get this ID in the response of the Task POST endpoint example: “07131248-1535-0216-1000-17384017ad04”
limitintegerНеобязательный
the maximum number of returned pages optional field default value: 100 maximum value: 1000
offsetintegerНеобязательный
offset in the results array of returned pages optional field default value: 0 if you specify the 10 value, the first ten pages in the results array will be omitted and the data will be provided for the successive pages
filtersarrayНеобязательный
array of results filtering parameters optional field you can add several filters at once (8 filters maximum) you should set a logical operator and, or between the conditions the following operators are supported: regex, not_regex, , , >, >=, =, , in, not_in, like, not_like you can use the % operator with like and not_like to match any string of zero or more characters example: ["reason","=","robots_txt"][["reason","","robots_txt"], "and", ["url","not_like","%/wp-admin/%"]] [["url","not_like","%/wp-admin/%"], "and", [["reason","","meta_tag"],"or",["reason","","http_header"]]] The full list of possible filters is available by this link.

Пример запроса

curl -X POST "https://openapi.felo.ai/v1/beta/dataforseo/on_page/non_indexable" \
  -H "Authorization: Bearer $FELO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "id": "<string>",
  "limit": 0,
  "offset": 0,
  "filters": "<string>"
}'

Ответ

Поля ответа

versionstringНеобязательный
the current version of the API
status_codeintegerНеобязательный
general status code you can find the full list of the response codes here Note: we strongly recommend designing a necessary system for handling related exceptional or error conditions
status_messagestringНеобязательный
general informational message you can find the full list of general informational messages here
timestringНеобязательный
execution time, seconds
costnumberНеобязательный
total tasks cost, USD
tasks_countintegerНеобязательный
the number of tasks in the tasks array
tasks_errorintegerНеобязательный
the number of tasks in the tasks array returned with an error
tasksstring[]Необязательный
array of tasks
idstringНеобязательный
task identifier unique task identifier in our system in the UUID format
status_codeintegerНеобязательный
status code of the task generated by DataForSEO; can be within the following range: 10000-60000 you can find the full list of the response codes here
status_messagestringНеобязательный
informational message of the task you can find the full list of general informational messages here
timestringНеобязательный
execution time, seconds
costnumberНеобязательный
cost of the task, USD
result_countintegerНеобязательный
number of elements in the result array
pathstring[]Необязательный
URL path
dataobjectНеобязательный
contains the same parameters that you specified in the POST request
resultstring[]Необязательный
array of results
crawl_progressstringНеобязательный
status of the crawling session possible values: in_progress, finished
crawl_statusobjectНеобязательный
details of the crawling session
max_crawl_pagesintegerНеобязательный
maximum number of pages to crawl indicates the max_crawl_pages limit you specified when setting a task
pages_in_queueintegerНеобязательный
number of pages that are currently in the crawling queue
pages_crawledintegerНеобязательный
number of crawled pages
total_items_countintegerНеобязательный
total number of relevant items in the database
items_countintegerНеобязательный
number of items in the results array
itemsstring[]Необязательный
items array
reasonstringНеобязательный
the reason why the page is non-indexable can take the following values: robots_txt, meta_tag, http_header, attribute, too_many_redirects
urlstringНеобязательный
url of the non-indexable page