This used to happen before we disabled some tests:
{noformat}
HSEARCH400007: Elasticsearch request failed: HSEARCH400090: Elasticsearch response indicates a failure.
Request: POST /indexname/_search with parameters {size=10000, routing=someText_8,someText_9,someText_6,someText_7,someText_4,someText_5,someText_2,someText_3,someText_0,someText_1}
Response: 403 'Forbidden' with body
{
"message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
}
{noformat}
I wasn't able to pinpoint the actual problem. Everything is fine as long as we use only one routing key, but as soon as we used multiple keys, all hell breaks loose...
I checked that the signing library apparently works fine; or at least it behaves similarly to the official python library for this specific request.
To debug:
* enable full logging of requests/responses in Apache HTTP Client:
{noformat}
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog
-Dorg.apache.commons.logging.simplelog.showdatetime=true
-Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG
-Dorg.apache.commons.logging.simplelog.log.org.apache.http.wire=ERROR
{noformat}
* You'll see an aws request id in the response to the request that failed. Maybe we can find more information about what went wrong in Amazon Athena by querying the logs: https://us-east-1.console.aws.amazon.com/athena/home?force&force=®ion=us-east-1#query |
|
