Data Schemas Reference

This reference provides detailed documentation of all data schemas used in the Vysion API responses. Understanding these schemas is essential for proper data parsing and integration.

Core Response Structure

All Vysion API responses follow a consistent wrapper format:

{
  "data": {
    "total": 0,
    "hits": []
  },
  "error": null
}

Field	Type	Description
`data`	object/null	Contains the response data or null if error occurred
`data.total`	integer	Total number of results available
`data.hits`	array	Array of result objects
`error`	object/null	Error information or null if successful

Document Schemas

DocumentHit Schema

Used in document search and retrieval endpoints.

{
  "page": {
    "id": "string",
    "url": {
      "url": "string",
      "networkProtocol": "string",
      "domainName": "string",
      "port": 0,
      "path": "string",
      "signature": "uuid",
      "network": "tor"
    },
    "foundAt": "string",
    "pageTitle": "string",
    "language": "en",
    "html": "string",
    "text": "string",
    "sha1sum": "string",
    "sha256sum": "string",
    "ssdeep": "string",
    "detectionDate": "2019-08-24T14:15:22Z",
    "ingestionDate": "2019-08-24T14:15:22Z",
    "screenshot": "string",
    "chunk": false,
    "htmlOversize": false,
    "docType": "text/html"
  },
  "tag": [
    {
      "namespace": "string",
      "predicate": "string",
      "value": "string"
    }
  ],
  "email": [],
  "paste": [],
  "skype": [],
  "telegram": [],
  "whatsapp": [],
  "bitcoin_address": [
    {
      "value": "string"
    }
  ],
  "polkadot_address": [],
  "ethereum_address": [],
  "monero_address": [],
  "ripple_address": [],
  "zcash_address": []
}

Page Object

Field	Type	Required	Description
`id`	string	true	Unique document identifier
`url`	object	true	URL information object
`foundAt`	string	false	Source where document was found
`pageTitle`	string	false	Title of the webpage
`language`	string	false	Detected language (ISO 639-1)
`html`	string	false	Raw HTML content
`text`	string	false	Extracted text content
`sha1sum`	string	false	SHA1 hash of content
`sha256sum`	string	false	SHA256 hash of content
`ssdeep`	string	false	Fuzzy hash for similarity detection
`detectionDate`	string	true	ISO 8601 timestamp of crawl/capture date
`ingestionDate`	string	false	ISO 8601 timestamp of when the page entered the Vysion index
`screenshot`	string	false	Screenshot URL if available
`chunk`	boolean	false	Whether this is a partial document
`htmlOversize`	boolean	false	`true` when full HTML exceeds the inline size limit and is offloaded to S3
`docType`	string	false	MIME type inferred from the URL path (e.g. `text/html`, `application/pdf`)

URL Object

Field	Type	Description
`url`	string	Complete URL
`networkProtocol`	string	Protocol (http, https)
`domainName`	string	Domain name
`port`	integer	Port number
`path`	string	URL path
`signature`	string	URL signature (UUID)
`network`	string	Network type (tor, clearnet)

Tag Object

Field	Type	Description
`namespace`	string	Tag namespace
`predicate`	string	Tag predicate
`value`	string	Tag value

Ransomware Schemas

RansomwareHit Schema

Used for ransomware victim data.

{
  "page": {
    "id": "string",
    "url": {
      "url": "string",
      "networkProtocol": "string",
      "domainName": "string",
      "port": 0,
      "path": "string",
      "signature": "uuid",
      "network": "tor"
    },
    "foundAt": "string",
    "pageTitle": "string",
    "language": "en",
    "detectionDate": "2019-08-24T14:15:22Z",
    "ingestionDate": "2019-08-24T14:15:22Z",
    "htmlOversize": false,
    "docType": "text/html"
  },
  "tag": [],
  "ransomwareGroup": "string",
  "companyName": "string",
  "companyAddress": "string",
  "companyLink": "string",
  "country": "string",
  "naics": "string",
  "industry": "string"
}

Field	Type	Description
`page`	object	Page information (see DocumentHit)
`tag`	array	Associated tags
`ransomwareGroup`	string	Name of ransomware group
`companyName`	string	Victim company name
`companyAddress`	string	Company address
`companyLink`	string	Company website
`country`	string	Country code
`naics`	string	NAICS industry code
`industry`	string	Industry description

Stat Schema

Used for statistics endpoints.

{
  "key": "string",
  "doc_count": 0
}

Field	Type	Description
`key`	string	Category identifier
`doc_count`	integer	Number of occurrences

AggStats Schema

Used for aggregated statistics with sub-categories.

{
  "key": "string",
  "doc_count": 0,
  "key_as_string": "string",
  "agg": {
    "buckets": [
      {
        "key": "string",
        "doc_count": 0
      }
    ]
  }
}

Field	Type	Description
`key`	string	Primary category key
`doc_count`	integer	Total count for category
`key_as_string`	string	Human-readable key
`agg.buckets`	array	Sub-category breakdowns

Instant Messaging Schemas

ImMessageHit Schema

Used for Telegram and Discord message data.

{
  "userId": 0,
  "username": "string",
  "channelId": 0,
  "messageId": "string",
  "message": "string",
  "channelTitle": "string",
  "topicId": 0,
  "topicTitle": "string",
  "replyToMessageId": 0,
  "isForumTopic": false,
  "languages": [
    {
      "language": "string",
      "probability": 0.95
    }
  ],
  "sha1sum": "string",
  "sha256sum": "string",
  "media": "string",
  "detectionDate": "2019-08-24T14:15:22Z",
  "serverId": "string",
  "serverTitle": "string",
  "platform": "telegram"
}

Field	Type	Description
`userId`	integer/string	User identifier
`username`	string	Username on platform
`channelId`	integer/string	Channel/group identifier
`messageId`	string	Unique message ID
`message`	string	Message content
`channelTitle`	string	Channel/group name
`topicId`	integer/string	Forum topic ID (Telegram forum-enabled groups only)
`topicTitle`	string	Forum topic title (Telegram forum-enabled groups only)
`replyToMessageId`	integer/string	ID of the message this is a reply to (Telegram only)
`isForumTopic`	boolean	`true` when the message belongs to a forum topic (Telegram only)
`languages`	array	Detected languages with confidence
`sha1sum`	string	Message content hash
`sha256sum`	string	Message content hash
`media`	string	Media type if present
`detectionDate`	string	ISO 8601 timestamp
`serverId`	integer/string	Discord server ID (Discord only)
`serverTitle`	string	Discord server name (Discord only)
`platform`	string	Platform name

ImProfileHit Schema

Used for user profile data.

{
  "userId": 0,
  "usernames": ["string"],
  "firstName": ["string"],
  "lastName": ["string"],
  "detectionDate": "2019-08-24T14:15:22Z",
  "profilePhoto": ["string"],
  "bot": false,
  "discordLink": ["string"],
  "discriminator": [0],
  "platform": "discord",
  "email": [{"value": "string"}],
  "telegram": [{"value": "string"}],
  "whatsapp": [{"value": "string"}],
  "bitcoin_address": [{"value": "string"}],
  "ethereum_address": [{"value": "string"}],
  "monero_address": [{"value": "string"}],
  "ripple_address": [{"value": "string"}],
  "zcash_address": [{"value": "string"}],
  "polkadot_address": [{"value": "string"}]
}

Field	Type	Description
`userId`	integer	User identifier
`usernames`	array	Known usernames
`firstName`	array	Known first names
`lastName`	array	Known last names
`detectionDate`	string	ISO 8601 timestamp
`profilePhoto`	array	Profile photo URLs
`bot`	boolean	`true` if the account is a bot
`discordLink`	array	Discord invite/profile links associated with the user
`discriminator`	array	Discord legacy discriminators (4-digit tags)
`platform`	string	Platform name
`email`	array	Associated email addresses
`telegram`	array	Telegram handles
`whatsapp`	array	WhatsApp numbers
`*_address`	array	Cryptocurrency addresses

ImChannelHit Schema

Used for channel/group information.

{
  "channelId": 0,
  "channelTitles": ["string"],
  "detectionDate": "2019-08-24T14:15:22Z",
  "creationDate": "2019-08-24T14:15:22Z",
  "channelPhoto": ["string"],
  "hasTopics": true,
  "topics": [
    {
      "id": 1,
      "title": "General",
      "topMessage": 42,
      "closed": false,
      "pinned": true,
      "hidden": false
    }
  ],
  "serverId": "string",
  "serverTitle": ["string"],
  "platform": "telegram"
}

Field	Type	Description
`channelId`	integer	Channel identifier
`channelTitles`	array	Known channel names
`detectionDate`	string	When detected
`creationDate`	string	Channel creation date
`channelPhoto`	array	Channel photo URLs

ImServerHit Schema

Used for Discord server information.

{
  "serverId": 0,
  "serverTitles": ["string"],
  "detectionDate": "2019-08-24T14:15:22Z",
  "creationDate": "2019-08-24T14:15:22Z",
  "serverPhoto": ["string"],
  "memberCount": 0,
  "discordLink": ["string"]
}

Field	Type	Description
`serverId`	integer	Server identifier
`serverTitles`	array	Known server names
`detectionDate`	string	When detected
`creationDate`	string	Server creation date
`serverPhoto`	array	Server icon URLs
`memberCount`	integer	Number of members
`discordLink`	array	Discord invite links

Leak Schemas

LeakHit Schema

Used for leaked data from Telegram channels.

{
  "id": "string",
  "detectionDate": "2024-01-15T10:30:00Z",
  "filePath": "leaked_database.sql",
  "fileHash": "a3b2c1d4e5f6...",
  "fileSize": 1024000,
  "fileType": "sql",
  "detectedMimeType": "text/plain",
  "decompressedFilename": "leaked_data.pdf",
  "archiveSource": "archive.zip",
  "archiveMemberPath": "leaked_data.pdf",
  "detectedInfo": {
    "emails": ["user@example.com"],
    "usernames": ["johndoe"],
    "phone_numbers": ["+1234567890"],
    "ipv4_addresses": ["192.168.1.1"],
    "ipv6_addresses": ["2001:0db8::1"],
    "bitcoin_addresses": ["1A1zP1eP..."],
    "ethereum_addresses": ["0x742d35..."],
    "hashes": ["a3b2c1d4e5f6..."]
  },
  "telegram": {
    "telegram_id": "-1002104057089_108",
    "channelId": -1002104057089,
    "messageId": 108,
    "channelName": "Data Leaks",
    "channelUsername": "dataleaks"
  },
  "language": "en",
  "languages": [
    {
      "language": "en",
      "probability": 0.95
    }
  ],
  "parseStatus": "success",
  "downloadUrl": "string",
  "highlight": {
    "detectedInfo.emails": ["<mark>user@example.com</mark>"],
    "content": ["snippet 1", "snippet 2"]
  }
}

Core Fields

Field	Type	Required	Description
`id`	string	true	Unique leak identifier
`detectionDate`	string	true	ISO 8601 timestamp
`filePath`	string	false	Path to leaked file
`fileHash`	string	false	File hash (SHA256/SHA1/MD5)
`fileSize`	integer	false	File size in bytes
`fileType`	string	false	File extension/type
`detectedMimeType`	string	false	MIME type from file content

Archive Fields

Field	Type	Description
`decompressedFilename`	string	Original filename if from archive
`archiveSource`	string	Parent archive filename
`archiveMemberPath`	string	Path within archive

Detected Information

The detectedInfo object contains extracted entities:

Field	Type	Description
`emails`	array	Email addresses found
`usernames`	array	Usernames found
`phone_numbers`	array	Phone numbers found
`ipv4_addresses`	array	IPv4 addresses found
`ipv6_addresses`	array	IPv6 addresses found
`bitcoin_addresses`	array	Bitcoin wallet addresses
`ethereum_addresses`	array	Ethereum wallet addresses
`monero_addresses`	array	Monero wallet addresses
`ripple_addresses`	array	Ripple wallet addresses
`zcash_addresses`	array	Zcash wallet addresses
`polkadot_addresses`	array	Polkadot wallet addresses
`binance_addresses`	array	Binance Coin addresses
`dash_addresses`	array	Dash wallet addresses
`hashes`	array	File hashes found in content

Telegram Object

Field	Type	Description
`telegram_id`	string	Telegram message identifier
`channelId`	integer	Channel ID
`messageId`	integer	Message ID
`channelName`	string	Channel name
`channelUsername`	string	Channel username

Optional Fields

Field	Type	When Present	Description
`downloadUrl`	string	`/leak/{id}` only	Presigned S3 download URL
`highlight`	object	`/leak/search` only	Highlighted search matches
`language`	string	When detected	Primary language code
`languages`	array	When detected	Languages with confidence
`parseStatus`	string	Always	Parsing status

Feed Schemas

RansomFeedHit Schema

Used in daily ransomware feeds.

{
  "id": "string",
  "companyName": "string",
  "companyLink": "string",
  "url": "string",
  "ransomwareGroup": "string",
  "detectionDate": "2019-08-24T14:15:22Z",
  "text": "string",
  "country": "string",
  "naics": "string",
  "industry": "string"
}

Field	Type	Description
`id`	string	Unique feed entry identifier (required)
`companyName`	string	Victim company name
`companyLink`	string	Company website
`url`	string	Ransomware leak-site URL (required)
`ransomwareGroup`	string	Name of ransomware group (required)
`detectionDate`	string	ISO 8601 timestamp (required)
`text`	string	Excerpt of the ransom post text
`country`	string	Country code
`naics`	string	NAICS industry code
`industry`	string	Industry description

ImFeedHit Schema

Used in Telegram channel feeds.

{
  "id": "string",
  "telegram": ["string"],
  "detectionDate": "2019-08-24T14:15:22Z",
  "url": "string",
  "path": "string",
  "network": "string"
}

OnionFeedHit Schema

Used in the daily onion (Tor hidden service) feed.

{
  "id": "string",
  "url": {
    "url": "string",
    "networkProtocol": "string",
    "domainName": "string",
    "port": 0,
    "path": "string",
    "signature": "uuid",
    "network": "tor"
  },
  "path": "string",
  "detectionDate": "2019-08-24T14:15:22Z",
  "tag": [
    {
      "namespace": "string",
      "predicate": "string",
      "value": "string"
    }
  ]
}

Field	Type	Description
`id`	string	Unique feed entry identifier (required)
`url`	object	URL object (see DocumentHit → `url`)
`path`	string	Path component of the URL (required)
`detectionDate`	string	ISO 8601 timestamp (required)
`tag`	array	Associated tags

CryptoFeedHit Schema

Used in the wallet feed for clearnet/darknet pages mentioning cryptocurrency addresses.

{
  "id": "string",
  "url": "string",
  "detectionDate": "2019-08-24T14:15:22Z",
  "network": "tor",
  "title": "string",
  "tag": [
    {
      "namespace": "string",
      "predicate": "string",
      "value": "string"
    }
  ],
  "bitcoin_address": [{"value": "string"}],
  "ethereum_address": [{"value": "string"}],
  "monero_address": [{"value": "string"}],
  "ripple_address": [{"value": "string"}],
  "zcash_address": [{"value": "string"}],
  "polkadot_address": [{"value": "string"}]
}

Field	Type	Description
`id`	string	Unique feed entry identifier (required)
`url`	string	Source URL
`detectionDate`	string	ISO 8601 timestamp (required)
`network`	string	Network where the page was found (`tor`, `clearnet`, …) (required)
`title`	string	Page title
`tag`	array	Associated tags
`*_address`	array	Cryptocurrency addresses detected on the page

CryptoTelegramFeedHit Schema

Used in the wallet feed when the source is a Telegram profile.

{
  "userId": "string",
  "usernames": ["string"],
  "firstName": ["string"],
  "lastName": ["string"],
  "detectionDate": "2019-08-24T14:15:22Z",
  "profilePhoto": ["string"],
  "bitcoin_address": [{"value": "string"}],
  "ethereum_address": [{"value": "string"}],
  "monero_address": [{"value": "string"}],
  "ripple_address": [{"value": "string"}],
  "zcash_address": [{"value": "string"}],
  "polkadot_address": [{"value": "string"}]
}

Field	Type	Description
`userId`	string	Telegram user identifier (required)
`usernames`	array	Known usernames
`firstName`	array	Known first names
`lastName`	array	Known last names
`detectionDate`	string	ISO 8601 timestamp (required)
`profilePhoto`	array	Profile photo URLs
`*_address`	array	Cryptocurrency addresses associated with the profile

CryptoDiscordFeedHit Schema

Used in the wallet feed when the source is a Discord profile.

{
  "userId": "string",
  "usernames": ["string"],
  "creationDate": "2019-08-24T14:15:22Z",
  "detectionDate": "2019-08-24T14:15:22Z",
  "profilePhoto": ["string"],
  "bitcoin_address": [{"value": "string"}],
  "ethereum_address": [{"value": "string"}],
  "monero_address": [{"value": "string"}],
  "ripple_address": [{"value": "string"}],
  "zcash_address": [{"value": "string"}],
  "polkadot_address": [{"value": "string"}]
}

Field	Type	Description
`userId`	string	Discord user identifier (required)
`usernames`	array	Known usernames
`creationDate`	string	Account creation date
`detectionDate`	string	ISO 8601 timestamp (required)
`profilePhoto`	array	Profile photo URLs
`*_address`	array	Cryptocurrency addresses associated with the profile

Common Data Types

Language Object

{
  "language": "en",
  "probability": 0.95
}

Field	Type	Description
`language`	string	ISO 639-1 language code
`probability`	number	Confidence score (0-1)

Address Object

{
  "value": "string"
}

Used for all cryptocurrency addresses and contact information.

Error Object

{
  "code": 400,
  "message": "string"
}

Field	Type	Description
`code`	integer	HTTP status code
`message`	string	Error description

ValidationError Object

{
  "detail": [
    {
      "loc": ["string"],
      "msg": "string",
      "type": "string"
    }
  ]
}

Field	Type	Description
`detail`	array	Array of validation errors
`detail[].loc`	array	Field location path
`detail[].msg`	string	Error message
`detail[].type`	string	Error type

Enumerated Values

Network Types

Value	Description
`tor`	Tor hidden service
`clearnet`	Standard internet

Language Codes

The API uses ISO 639-1 language codes. Common values include:

Code	Language
`en`	English
`es`	Spanish
`fr`	French
`de`	German
`ru`	Russian
`zh`	Chinese
`ar`	Arabic

Platform Types

Value	Description
`telegram`	Telegram messaging
`discord`	Discord messaging

Schema Validation

All schemas are validated server-side. Common validation rules include:

Required fields: Must be present in the response
Type validation: Fields must match specified types
Format validation: Dates must be ISO 8601, UUIDs must be valid format
Range validation: Numeric fields may have min/max constraints

Best Practices

Handling Optional Fields

# Safe field access
def get_document_title(document_hit):
    page = document_hit.get('page', {})
    return page.get('pageTitle', 'Unknown Title')

Type Checking

# Validate data types
def validate_message_hit(hit):
    required_fields = ['userId', 'channelId', 'messageId', 'detectionDate']
    for field in required_fields:
        if field not in hit:
            raise ValueError(f"Missing required field: {field}")

Date Parsing

from datetime import datetime

def parse_detection_date(date_string):
    return datetime.fromisoformat(date_string.replace('Z', '+00:00'))