यह पृष्ठ अभी आपकी भाषा में उपलब्ध नहीं है। आप अंग्रेज़ी पाठ पढ़ रहे हैं।
Glossary
Web scraping
Web scraping is reading data out of a page the way a visitor sees it and keeping it as your own records.
How a scrape is put together
Every scrape is the same three moves in a loop. A browser opens an address, a piece of the page is located, and what it holds is written down under a name you chose.
- 1
Open the address
A real browser loads the page, so anything drawn by scripts is there to be read.
- 2
Point at the element
A selector names the piece you mean, and the site decides whether it still matches tomorrow.
- 3
Store it under a name
The text becomes a value later steps can use, and that is what turns browsing into collecting.
What the page reads back about you
These are read in your browser on this page, and a site you collect from can read the same ones. The automation flag is the row most scraping tools leave switched on without noticing.
आपकी मशीन, अभी इसी क्षण
- Browser string
- आपकी मशीन पर माप जारी है
- Automation flag
- आपकी मशीन पर माप जारी है
- Languages
- आपकी मशीन पर माप जारी है
- Time zone
- आपकी मशीन पर माप जारी है
आपके ब्राउज़र में पढ़ा गया और यहाँ दिखाया गया। कहीं भी नहीं भेजा जाता।
Reading steps in a flow
The automation builder holds a fixed set of step kinds, and four of them read rather than act. They run inside a profile's own browser, with its fingerprint, cookies and proxy.
Extract text
Takes the first element a selector matches and keeps its text in a variable, trimmed and capped in length.
For each element
Runs its body once per matched element, up to a cap you set, and reads the list again on every pass.
Run JavaScript
Runs your own code in the page and stores what it returns, for when a selector is not enough.
Wait for request
Holds until the page asks for an address you name, so a late list is read after it lands.
Where the collected text ends up
This is the part worth knowing before you plan a job. Values live inside the run while it happens, and what survives afterwards is a log rather than a dataset.
- During the run
- The value sits in a variable and later steps can use it
- After the run
- It stays in the run log on this machine, newest runs kept
- Screenshots
- Files in a folder that only your own account can read
- A spreadsheet
- Not written: nothing here exports rows to a file for you
The local start route answers with a run identifier, never with the text that was read.
Often confused with
An API
A published interface gives you records and a contract. Scraping gives you whatever the page happened to render today.
Crawling
Crawling is finding the pages worth visiting. Scraping is reading a page you already chose, and a real job needs both.
A headless browser
Running with no window is one way to scrape, not a kind of scraping, and it is the part a site notices first.
Why a scrape gets noticed
A broken selector fails loudly and is fixed in a minute. Everything below fails quietly.
One address many reads
A datacentre address pulling a thousand pages an hour is the oldest pattern in the field.
A rhythm no person keeps
Even pauses, identical scroll distances and a run that starts on the same second every day.
One browser for everything
Twenty accounts behind one fingerprint and one cookie jar are tied together before the first page is read.
What this does not do
- There is no data export. What a reading step collects lives in the run and in the run log on this machine, and nothing writes a spreadsheet or a database for you.
- The local start route hands back a run identifier, not the rows. A program on this machine can list flows and start one, and no call returns what was read.
- A reading step takes the first element its selector matches and the text inside it. There is no table understanding, no paging through results and no schema of any kind.
- None of this fixes the part that ends jobs. Address reputation, session history and the hour you work at close more accounts than selectors ever do.
Check it
Each claim here is a file you can open and a stand that runs it.
- The step kinds a flow may hold, and what a reading step keeps
- apps/desktop/src/lib/rpa.js
- A shared flow cannot be pointed at a file on your own disk
- apps/desktop/src/lib/rpa.js · apps/desktop/test/rpa-reach.js
- A run really drives the browser and really changes the page
- apps/desktop/test/сценарий-живьём.js
- What a finished run keeps on this machine, and what it drops
- apps/desktop/src/main/main.js · apps/desktop/test/run-log-survives-restart.js
Questions
- Is web scraping legal?
- That depends on the site, the data and where you are. Terms of service, database rights and personal data rules all apply before any technical question does.
- Why does it work by hand and fail in a script?
- Usually because the script announces itself. A plain automation driver leaves the automation flag switched on and a default browser string in place, and both are read on the first request.
- Do I need a real browser at all?
- For a page that arrives finished, no. For anything drawn by scripts after load, a browser is the cheapest way to see what a visitor sees.
- Can one profile do all the reading?
- It can, and then everything you read is tied to one identity. Work that must not be linked belongs in separate profiles with separate exit addresses.
- How do I get the values out of ARMANOS?
- The run log on this machine is where they are, and the app puts them nowhere else. There is no export button for run results.
Read a page from a profile of its own
The builder, the run log and the profiles live in the app, and nothing about a flow leaves your computer unless you send it.