Atriis - Supplier Latency - Amadeus – Incident details

Supplier Latency - Amadeus

Resolved
Degraded performance
Started 23 days agoLasted 3 days

Affected

API

Operational from 2:50 PM to 12:41 PM

Hotel Search API

Operational from 2:50 PM to 12:41 PM

Atriis Core

Degraded performance from 2:50 PM to 6:46 AM, Operational from 6:46 AM to 8:46 AM, Degraded performance from 8:46 AM to 9:46 AM, Operational from 9:46 AM to 12:41 PM

Updates
  • Resolved
    Resolved

    Hi,

    All connections are back to normal.

    Regards, Atriis Support

  • Monitoring
    Monitoring

    Hi,

    We are able to get Amadeus and NDC results now, it appears that the services are back and active.

    We are monitoring the behavior for now, hopefully this is fully resolved now.

  • Update
    Update

    Hi,

    It appears that Amadeus have yet to resolve the issue, and we're also encountering issues with NDC content (which is based on Amadeus). We're waiting to get an update from Amadeus and will inform you once we have one.

    Atriis Support

  • Identified
    Identified

    Hi,

    It appears Amadeus are again encountering issues with master pricer searches. We'll try to get an update from them.

    Regards, Atriis Support

  • Resolved
    Resolved

    Hi,

    It appears that Amadeus Master Pricer is back up and running, from testing and from our logs we no longer see issues.

    Below is their latest update.

    Moved 100% of MP traffic to SDC and away from blackhole COMPLETE Moved Flex to healthy regions (currently 55% FC, 40% Erding, 5% NE) COMPLETE Scale out CB cluster in Sweden COMPLETE Increase connections to CB in Sweden - COMPLETE Scale out CB cluster in Germany COMPLETE Increase connections to CB cluster in Germany COMPLETE Increased capacity to North Europe to process more MP & SOS traffic COMPLETE NO MORE INCREASE POSSIBLE HERE

    Regards, Atriis Support

  • Update
    Update

    Hi,

    Unfortunately although we're getting a response in certain cases, we're still seeing timeout responses from Amadeus, it appears that there's still a connection overload on their end. We'll advise once we have further updates.

  • Monitoring
    Monitoring

    Hi,

    Even though informed by Amadeus that they resolved the issue, it appears to not be fully restored on all Office IDs, we've asked for further clarification on this.

  • Resolved
    Resolved

    Hi,

    Amadeus have advised that the issue has been resolved on their end, see further details below.

    Please advise in case you still encounter any issue with flight searches.

    AUTO-CASCADED FROM MIR 36246463:

    Dear Customer,

    On the 26th of August, multiple customers and monitoring alarms raised

    critical incidents for Shopping services shortly after 11:00GMT.

    Customers may have experienced errors or response timeouts while

    conducting shopping or pricing transactions via shopping services;

    specifically, Flex Pricer and Master Pricer through the Amadeus Digital

    Commerce channels.

    Investigations revealed an issue with the related clusters, where

    traffic was initially lost in the data center region. Amadeus support

    teams initially rerouted the traffic away from the affected server and

    redistributed across the remaining data center regions, subsequently

    increasing server resources and recycling application processes to

    improve performance stability. These actions resulted in the initial

    recovery of the incident at 12:26 GMT. However, at 13:27 GMT, a

    reoccurrence was observed as the issue began to affect the remaining

    regions.

    Amadeus support teams performed multiple recovery actions in attempt to

    make the systems stable including recycling the affected servers,

    scaling up PODS, re-balanced clusters, falling back any recent

    implementation and also deactivate Master Pricer. Amadeus support teams

    reactivated the Flex Pricer traffic in related regions recovering the

    impact on Flex Pricer. Master Pricer traffic was gradually reactivated

    to avoid any sudden spikes and ensure stability. After the process was

    completed, positive outcomes have been observed, with some customers

    confirming recovery. Environments will remain under heightened

    monitoring, at the moment this incident has been recovered at 00:37 GMT.

    Regards,

    Amadeus Support Center - Support and OPS Analysts.

  • Monitoring
    Monitoring

    Hi,

    We've been informed of the following from Amadeus:

    Amadeus support teams reactivated a minor portion of traffic for Master Pricer which is now visible. Support teams will gradually incease the traffic and continue to monitor the situation.

    Support teams at Amadeus are also making preparations to enhance the capacity of additional data centers, with plans to activate Master Pricer in those locations as well. Further updates will be provided as soon as they become available.

    Further updates to follow.

  • Update
    Update

    Hi,

    Unfortunately, the issue still persists, and Amadeus has not yet provided a solution. We’ve escalated this with their team, and it appears to be a global error.

  • Identified
    Identified

    Hi,

    We've been informed of latency issue with Amadeus
    You may experience issues with searching, booking or ticketing for this specific supplier.

    We'll update once supplier operations are back to normal