You are here: Home > Analyse > Internet Measurements > RIPE Atlas > Documentation > RIPE Atlas Result Streams

RIPE Atlas Result Streams

RIPE Atlas has been providing downloadable results from the very beginning of the project. This works well if you know what time frame you're interested in, and want to get to the data collected at that time.

The streaming data service allows you to tap into the real-time data flow of all the collected public results. Every time our system receives a data point or a probe connectivity event occurs, it's also delivered to the clients that are "tuned in" to that result stream. This feature is implemented using web sockets.

Highlights

  • This service is in prototype status. We're observing how our system reacts to the streams provided to users in order to evaluate the feasibility and usefulness of a production service.
  • Streaming uses the Socket.IO protocol over websockets for real-time event-based communication
  • You can only subscribe ("tune in") to results delivered by public measurements.
  • You can subscribe to the connectivity events of any probe.
  • We're inviting our community to check out the gallery of visualisations provided by the RIPE Atlas team and our users as soon as it becomes available, and to come up with new visualisations and/or to enhance the existing ones.

It is also possible to stream historic events at the rate at which they occurred, or with a speedup factor.

Basic Examples

The service address for the stream is http://atlas-stream.ripe.net.

The following examples will show how to connect to the streaming for the various stream types.

Stream type: result

When stream_type is set to "result", the client will receive measurement results.

<script src="https://atlas-stream.ripe.net/socket.io.js"></script>
<script>

    // Create a connection (it can be also http on port 80)
    var socket = io("https://atlas-stream.ripe.net:443", { path : "/stream/socket.io" });

    // Subscribe to results coming from all the probes involved in the measurement 1791207
    socket.emit("atlas_subscribe", { stream_type: "result", msm: 1791207 });

    // Declare a callback to be executed when a measurement result is received
    socket.on("atlas_result", function(result){
        console.log("I received ", result);
    });

</script>

Stream type: probestatus

When stream_type is set to "probestatus", the client will receive connection and disconnection events of probes.

<script src="https://atlas-stream.ripe.net/socket.io.js"></script>
<script>

    // Create a connection
    var socket = io("https://atlas-stream.ripe.net:443", { path : "/stream/socket.io" });

    // Subscribe to the connection events of the probe 22527
    socket.emit("atlas_subscribe", { stream_type: "probestatus", prb: 22527 });

    // Declare a callback to be executed when a probe connection event is received
    socket.on("atlas_probestatus", function(status){
        console.log("I received ", status);
    });

</script>

Stream type: metadata

When stream_type is set to "metadata", the client will receive measurements metadata every time a measurement is created or edited.

<script src="https://atlas-stream.ripe.net/socket.io.js"></script>
<script>

    // Create a connection
    var socket = io("https://atlas-stream.ripe.net:443", { path : "/stream/socket.io" });

    // Subscribe to measurement creation/edit events targeting a specific prefix
    socket.emit("atlas_subscribe", { stream_type: "metadata", destinationPrefix: "2001:67c:2e8::/48" });

    // Declare a callback to be executed when an event is received
    socket.on("atlas_metadata", function(metadata){
        console.log("I received ", metadata);
    });

</script>

Parameters

A list of all the parameters available in the streaming system will follow.

Subscription parameters

The following parameters can be used to specify what the client wants to receive on the channel. Some of those parameters are in common among the various stream_type, others are instead specific. See the tables below for more details.

Warning: if a client, or the network between the client and the server, is not able to keep up with the number of results streamed by the RIPE Atlas streaming, the socket will be put in volatile mode and an error raised. The volatile mode will drop all the samples not delivered in time to the client. To reduce the network overhead see the "buffering" parameter.

Common parameters (in addition to stream_type)

Name                                Description
prb A specific probe ID. If you don't set this parameter, you will receive results from all the probes
acceptedFields A list of accepted fields name, the messages will be pruned server side. If you don't set this parameter you will receive all the fields
enrichProbes If you want to enrich the information received with the "probestatus" stream about the probes (e.g. lat, long), set this option to true
equalsTo Allows to filter by values. E.g. with {status: "connected", asn: "3333|4444"} you will receive all the messages with a connected status and ASn equals to 3333 or 4444
lessThan Allows to filter by values. E.g. with {valueX: 15} you will receive all the messages with a valueX less than 15
greaterThan Allows to filter by values. E.g. with {valueX: 15} you will receive all the messages with a valueX greater than 15

Parameters for "result" stream_type

Name                                Description
msm A specific measurement ID
type Streams all the results of the specified type, i.g. ping, traceroute, ntp, http, dns, ssl
sourceAddress Streams all the results coming from a probe having the specified address
sourcePrefix Streams all the results coming from a probe having an address in the specified prefix
destinationAddress Streams all the results measuring the specified address
destinationPrefix Streams all the results measuring an address in the specified prefix
passThroughHost Streams all the traceroutes passing through the specified host. Only for traceroute measurements.
passThroughPrefix Streams all the traceroutes passing through a host in the specified prefix. Only for traceroute measurements.
sendBacklog Immediately fetch the last few minutes of results for a specific measurement ID. Mostly intended to cover the gap between the last available data in the REST API and the currently streamed results, and to recover results that might have been missed during brief disconnections.
buffering If set to true, the samples will be sent in bundles in order to reduce the network overhead. When this option is true, the socket is not volatile. False by default.

Parameters for playing historic events

You can replay events that have happened in the past, the various events will resemble the real timing. Note:

  • you cannot replay metadata;

  • you can replay only specific measurement IDs, and optionally filter by probe. Additional parameters will be ignored;

  • you cannot have multiple playbacks for the same measurement at the same time (e.g. one year ago and one month ago running concurrently). If you subscribe to a new probe for an already running measurement, you will simply start seeing results for that probe from the already-running measurement stream.

Name Description
startTimeStart time (seconds since 1970) of the event to replay.
stopTime (optional)End time (seconds since 1970) of the event to replay. For stopped measurements, defaults to the stop time. For other measurements, defaults to the present time.
speed (optional)The speedup factor (float). Defaults to 1.0 (original speed). There is a limit to the maximum speed which is determined by the number of participants for the given measurement.

Example:

<script src="https://atlas-stream.ripe.net/socket.io.js"></script>
<script>

    // Do something when we get a connection/disconnection
    socket.on("atlas_probestatus", function(status){
        console.log("I received ", status);
    });

    // Stream all connection/disconnection events from midnight on 2015-3-18
    socket.emit("atlas_subscribe", {
        stream_type: "probestatus",
        startTime: 1426633200
    });

</script>

Events that you can listen

Name                               Description
connect When the connection is established
disconnect When the connection is terminated
atlas_error Error messages coming from the stream service
atlas_result Measurement results of a "result" subscription
atlas_metadata Measurement metadata of a "metadata" subscription
atlas_probestatus Connection and disconnection event of a "probestatus" subscription
atlas_subscribed When a subscription to a channel is successful
atlas_unsubscribed When the client has been unsubscribed from a channel
atlas_replay_finished When replay of a historic stream has reached the end, according to the user given stopTime, the actual measurement stop time, or the present time

Events that you can emit

Name                               Description
atlas_subscribe To subscribe to a channel (parameters needed)
atlas_unsubscribe To unsubscribe from a channel (parameters needed)

You can also check out the source code of the visualisations below.

Available Visualisations