Skip to content

Commit

Permalink
leaksdb: document leaksdb/credentials/by_domain
Browse files Browse the repository at this point in the history
  • Loading branch information
aviau committed Sep 5, 2024
1 parent e28f867 commit fb9e66d
Show file tree
Hide file tree
Showing 4 changed files with 112 additions and 35 deletions.
7 changes: 6 additions & 1 deletion docs/api-reference/leaksdb/endpoints/get-by-domain.mdx
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
---
title: "List by Domain"
title: "List by Domain (deprecated)"
api: "GET https://api.flare.io/leaksdb/identities/by_domain/{domain}"
---

<Warning>
This API endpoint is deprecated and should be replaced by
[/by_domain_with_subdomains <Icon icon="code" size={16} />](/api-reference/leaksdb/endpoints/get-credentials-by-domain).
</Warning>

Returns a list of accounts matching the domain provided.
The format of the domain should include everything after the `@`.

Expand Down
77 changes: 77 additions & 0 deletions docs/api-reference/leaksdb/endpoints/get-credentials-by-domain.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: "List by Domain"
api: "GET https://api.flare.io/leaksdb/v2/credentials/by_domain_with_subdomains/{domain}"
---

Returns a list of accounts matching the domain provided.

Results are returned in ascending order by id.

<ResponseExample>

```json Response Example
{
"items": [
{
"domain": "scatterholt.com",
"hash": "B@dPassw0rd",
"id": 33880703907,
"identity_name": "[email protected]",
"imported_at": "2024-07-22T19:25:52.893439+00:00",
"known_password_id": null,
"source": {
"breached_at": null,
"description_en": "Collection of multiple combo lists (emails and passwords) exchanged on illicit networks.",
"description_fr": "Collection de multiples listes \"combos\" (adresses courriel et mots de passe) \u00e9chang\u00e9es sur des r\u00e9seaux illicites.",
"id": "combolists",
"is_alert_enabled": true,
"leaked_at": null,
"name": "Combolists"
},
"source_id": "combolists"
},
{
"domain": "scatterholt.com",
"hash": "1qaz2wsx",
"id": 33880703906,
"identity_name": "[email protected]",
"imported_at": "2024-07-22T19:25:52.893439+00:00",
"known_password_id": null,
"source": {
"breached_at": null,
"description_en": "Collection of multiple combo lists (emails and passwords) exchanged on illicit networks.",
"description_fr": "Collection de multiples listes \"combos\" (adresses courriel et mots de passe) \u00e9chang\u00e9es sur des r\u00e9seaux illicites.",
"id": "combolists",
"is_alert_enabled": true,
"leaked_at": null,
"name": "Combolists"
},
"source_id": "combolists"
}
],
"next": "WyJjb20uc2NhdHRlcmhvbHQiLCAxNjczNjg4ODg4NV0"
}
```

</ResponseExample>

## Paging

This endpoint supports the
[Flare standard paging pattern <Icon icon="book" size={16} />](/concepts/paging).

## Path Parameters

<ParamField path="domain" type="string" required>
The domain you want to lookup.
</ParamField>

## Query Parameters

<ParamField query="size" type="number">
Maximum size of the JSON object that will be returned (maximum 10 000)
</ParamField>

<ParamField query="from" type="number">
The `next` value from the last response.
</ParamField>
58 changes: 26 additions & 32 deletions docs/guides/credentials-export-domain.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: "Exporting a Domain's Credentials"
Flare's Leaked Credentials API can be used to export a domain's leaked credentials.

This guide will explain how to export all leaked credentials for the example.com domain using the
[leaksdb/identities/by_domain <Icon icon="code" size={16} />](/api-reference/leaksdb/endpoints/get-by-domain)
[credentials/by_domain <Icon icon="code" size={16} />](/api-reference/leaksdb/endpoints/get-credentials-by-domain)
endpoint.


Expand All @@ -15,21 +15,19 @@ endpoint.
<Steps>

<Step title="Fetch one page of results">
Use the `by_domains` endpoint to fetch one page of results.
Use the `credentials/by_domain` endpoint to fetch one page of results.

If this is the first page, you may omit the `from` parameter.

If the returned page of results is empty, this means that the export is complete.
</Step>

<Step title="Find the highest id in the response value">
The `by_domains` endpoint returns credentials grouped by identity.

This means that the highest id could be in any of the returned identities.
<Step title="Print the results">
Print the results by looping over the response's items.
</Step>

<Step title="Go back to step 1 until the response is empty">
Using the highest returned credential id, go back to step 1 to fetch the next page.
<Step title="Ratelimit and go to step 1">
Wait one second to avoid going over the API rate limit and to back to step 1 to fetch the next page.
</Step>

</Steps>
Expand Down Expand Up @@ -58,42 +56,38 @@ if not api_key:
api_client = FlareApiClient(api_key=api_key)

domain_name: str = "scatterholt.com"
from_: int = -1
from_: str | None = None

writer = csv.DictWriter(
sys.stdout,
fieldnames=[
"identity",
"password",
"source",
],
)

while True:
for resp in api_client.scroll(
method="GET",
url=f"/leaksdb/v2/credentials/by_domain/{domain_name}",
params={
"from": from_,
},
):
# Print results
credentials = resp.json()["items"]
for credential in credentials:
writer.writerow(
{
"identity": credential["identity_name"],
"password": credential["hash"],
"source": credential["source"]["id"],
},
)

# Rate limiting.
time.sleep(1)

# Fetch the next page
resp = api_client.get(
f"/leaksdb/identities/by_domain/{domain_name}",
params={
"from": from_,
},
)
# Stop on the first empty page.
identities = resp.json()
if not identities:
break

# Print all passwords
for identity in identities:
for password in identity["passwords"]:
from_ = max(from_, password["id"])
writer.writerow(
{
"identity": identity["name"],
"password": password["hash"],
},
)
```
</Accordion>

Expand Down
5 changes: 3 additions & 2 deletions docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -123,9 +123,10 @@
"group": "Leaked Credentials API",
"pages": [
{
"group": "Credentials Feed",
"group": "Credentials",
"pages": [
"api-reference/leaksdb/endpoints/get-credentials"
"api-reference/leaksdb/endpoints/get-credentials",
"api-reference/leaksdb/endpoints/get-credentials-by-domain"
]
},
{
Expand Down

0 comments on commit fb9e66d

Please sign in to comment.