Firehouse

title: Compliance Firehose API keywords: [“compliance firehose”, “firehose API”, “compliance API”, “enterprise firehose”, “compliance stream”, “firehose”] description: >Please note > >We have released a new compliance tool to X API v2 called batch compliance. This new tool allo…--- >Please note > >We have released a new compliance tool to X API v2 called batch compliance. This new tool allows you to upload large datasets of Post or user IDs to retrieve their compliance status in order to determine what data requires action in order to bring your datasets into compliance. >In addtion, both the batch compliance and the compliance firehose have been updated to support Post edits. For the compliance firehose, a new ‘tweet_edit’ event was added. See the Compliance Data Objects documentation for more details. Learn more about how Edit Post metadata works on the Edit Posts fundamentals page.

Overview

Enterprise One of X’s core values is to defend and respect the user’s voice. This includes respecting their expectations and intent when they delete, modify, or edit the content they choose to share on X. We believe that this is critically important to the long term health of one of the largest public, real-time information platforms in the world. X puts controls in the hands of its users, giving individuals the ability to control their own X experience. We believe that business consumers that receive X data have a responsibility to honor the expectations and intent of end users. For more information on the types of compliance events that are possible on the X platform, reference our article, Honoring User Intent on X. Any developer or company consuming X data via an API holds an obligation to use all reasonable efforts to honor changes to user content. This obligation extends to user events such as deletions, modifications, and changes to sharing options (e.g., content becoming protected or withheld). This also includes when users edit their Posts. Please reference the specific language in the Developer Policy and/or your X Data Agreement to understand how this obligation affects your use of X data. X offers the following solutions that deliver information about these user compliance events and whether a specific Post or User is publicly available or not. A brief overview of the solutions and their general integration patterns is detailed below:

GET statuses/lookup and GET users/lookup

* Format: REST API’s See: GET statuses/lookup and GET users/lookup. * These endpoints always return the latest version of any Post edits. All Post objects describing Posts created after the edit feature was introduced will include Post edit metadata. This is true even for Posts that were not edited. * For all Posts, requests for Posts more than 30 minutes after they were created will represent the final state of all Posts. * Deliver availability information for specific Posts or Users as defined by the caller as part of the API request. * May be used for ad-hoc spot-checking on the current availability state of a specific group of Posts / Users. * Ideal for customers who need a way to check the current state of a specific Post or User at a given moment in time. * These API’s provide a helpful mechanism that may be used by customers who need to check the availability of a piece of Content, for instance when: 1. Displaying Posts 2. Engaging with a Post(s) or User(s) in a 1:1 way 3. Distributing X Content to a 3rd party through an allowed file download 4. Storing Posts for extended periods of time

Compliance Firehose (enterprise only)

* Format: Streaming API See: Compliance Firehose. * Delivers realtime stream of Compliance activities on X. These activities include when Posts are edited. * May be used to maintain compliance state across a set of stored data as new compliance events happen on the platform. * Ideal for customers consuming and storing large quantities of X data for extended periods of time.

Guides

Compliance Best Practices

Recommendations & Best Practices

* Build Data Storage Schemas That Store Numeric Post ID and User ID: User messages require action to be taken on all Posts from that User. Therefore, since all compliance messages are delivered only by numeric ID, it is important to design storage schemas that maintain the relationship between Post and User based on numeric IDs. Data consumers will need to monitor compliance events by both Post ID and User ID and be able to update the local data store appropriately. * Build Schemas That Address All Compliance Statuses: Depending on how compliance activities will be addressed in various applications, it may be required to add other metadata to the data store. For instance, data consumers may decide to add metadata to an existing database to facilitate restricting the display of content in countries affected by a status_withheld message. * Handling Retweet Deletes: Retweets are a special kind of Post where the original message is nested in an object within the Retweet. In this case, there are two Post IDs referenced in a Retweet – the ID for the Retweet, and the ID for the original message (included in the nested object). When an original message is deleted, a Post delete message is issued for the original ID. Post deletion events typically trigger delete events for all Retweets. However, in some cases not all are sent and client systems should be tolerant of incomplete Retweet deletions. The deletion of the original ID should be sufficient to delete all subsequent Retweets. It is a best practice to reference the original Post ID when storing Retweets, and deleting all referenced Retweets when receiving Post delete events.

Compliance Data Objects

Compliance Firehose API

Possible types of compliance events will include Post (or “status”) events and User events, for which there are multiple types described below. Please note: * Read more about User statuses here and our developer policy around deleted Posts here. * The Compliance Firehose has been updated to provide ‘tweet_edit’ events. * Several User delete, protect and suspend events are not necessarily permanent and can toggle between states infinitely. These include: user_delete,user_undelete, user_protect, user_unprotect and user_suspend, user_unsuspend. * User_deletes are followed by status_deletes 30 days later only if the user has not selected to user_undelete their account. It is possible that a user_delete is reversed by the user and deletes for all of their Posts 30 days later do not occur. * User_suspend is an action that remains true unless the user is subject to an user_unsuspend event. These are not subject to any changes on a 30 day time period. Refer to the ‘Recommended Action’ column to understand how to process each type of event in order to respect the privacy and intent of the end user.

Original Message Type	Object	Permanent (Yes/No)	Recommended Action
delete	Status	Yes	Delete associated Post.
status_withheld	Status	Yes	Suppress associated Post in specific countries listed in the status_withheld message.
drop	Status	No	Remove the Post from public view.
undrop	Status	No	Status may be displayed again and treated as public.
tweet_edit	Status	Yes	Honor and, where relevant, display the new edit.
user_delete	User	No	Suppress or delete all Posts by associated user.
user_undelete	User	No	All Posts by associated user may be displayed again and treated as public.
user_protect	User	No	Suppress or delete all Posts by associated user.
user_unprotect	User	No	All Posts by associated user may be displayed again and treated as public.
user_suspend	User	No	Suppress or delete all Posts by associated user.
user_unsuspend	User	No	All Posts by associated user may be displayed again and treated as public.
scrub_geo	User	Yes	Delete all geodata provided by X for all Posts by the user prior to the specified Post in the scrub_geomessage. Note that subsequent Posts by a user may contain geodata that may be used.
user_withheld	User	Yes	Suppress Posts by associated user in specific countries listed in the user_withheld message.
delete	Favorite	Yes	Delete associated like/favorite.

Payload examples

See the payload examples below for each compliance event described in the table above. Post edit

{"tweet_edit":
   {
     "id": "1557445923210514432"
     "initial_tweet_id": "1557433858676740098",
     "edit_tweet_ids": ["1557433858676740098", "1557445923210514432"],
     "timestamp_ms": "1660155761384"
   }
 }

Post delete

{
  "delete": {
    "status": {
      "id": 601430178305220600,
      "id_str": "601430178305220608",
      "user_id": 3198576760,
      "user_id_str": "3198576760"
    },
    "timestamp_ms": "1432228155593"
  }
}

Post withheld

{
  "status_withheld": {
    "status": {
      "id": 601430178305220600,
      "id_str": "601430178305220608",
      "user_id": 3198576760,
      "user_id_str": "3198576760"
    },
    "withheld_in_countries": [
      "XY"
    ],
    "timestamp_ms": "1432228155593"
  }
}

Drop

{
  "drop": {
    "status": {
      "id": 601430178305220600,
      "id_str": "601430178305220600",
      "user_id": 3198576760,
      "user_id_str": "3198576760"
    },
    "timestamp_ms": "1432228155593"
  }
}

Undrop

{
  "undrop": {
    "status": {
      "id": 601430178305220600,
      "id_str": "601430178305220600",
      "user_id": 3198576760,
      "user_id_str": "3198576760"
    },
    "timestamp_ms": "1432228155593"
  }
}

Scrub geo

{
  "scrub_geo": {
    "user_id": 519761961,
    "up_to_status_id": 411552403083628540,
    "up_to_status_id_str": "411552403083628544",
    "user_id_str": "519761961",
    "timestamp_ms": "1432228180345"
  }
}

User delete

{
  "user_delete": {
    "id": 771136850,
    "timestamp_ms": "1432228153548"
  }
}

User undelete

{
  "user_undelete": {
    "id": 796250066,
    "timestamp_ms": "1432228149062"
  }
}

User withheld

{
  "user_withheld": {
    "user": {
      "id": 1375036644,
      "id_str": "1375036644"
    },
    "withheld_in_countries": [
      "XY"
    ],
    "timestampMs": "2014-08-27T23:49:41.839+00:00"
  }
}

User protect

{
  "user_protect": {
    "id": 3182003550,
    "timestamp_ms": "1432228177137"
  }
}

User unprotect

{
  "user_unprotect": {
    "id": 2911076065,
    "timestamp_ms": "1432228180113"
  }
}

User suspend

{
  "user_suspend": {
    "id": 3120539094,
    "timestamp_ms": "1432228194217"
  }
}

User unsuspend

{
  "user_unsuspend": {
    "id": 3293130873,
    "timestamp_ms": "1432228193828"
  }
}

integrating Compliance Firehose

The Compliance Firehose is a realtime streaming API that delivers compliance events that occur on the X platform. For an understanding of compliance events and how they are generated on X, please reference our article, [Honoring User Intent on X](/x-api/enterprise-gnip-2.0/fundamentals/f

Getting Started

Products

Fundamentals

Partners & Customers

Resources

Firehouse

Overview

GET statuses/lookup and GET users/lookup

Compliance Firehose (enterprise only)

Guides

Compliance Best Practices

Recommendations & Best Practices

Compliance Data Objects

Compliance Firehose API

Payload examples

Post delete

Post withheld

Drop

Undrop

Scrub geo

User delete

User undelete

User withheld

User protect

User unprotect

User suspend

User unsuspend

integrating Compliance Firehose

​Overview

​GET statuses/lookup and GET users/lookup

​Compliance Firehose (enterprise only)

​Guides

​Compliance Best Practices

​Recommendations & Best Practices

​Compliance Data Objects

​Compliance Firehose API

​Payload examples

Post delete

Post withheld

Drop

Undrop

​Scrub geo

User delete

User undelete

User withheld

User protect

User unprotect

User suspend

User unsuspend

​integrating Compliance Firehose

Overview

GET statuses/lookup and GET users/lookup

Compliance Firehose (enterprise only)

Guides

Compliance Best Practices

Recommendations & Best Practices

Compliance Data Objects

Compliance Firehose API

Payload examples

Scrub geo

integrating Compliance Firehose