Xero - beautiful accounting software

Xero Developer Help Center

Xero Developer Community

Community > API Authentication >

Failed to validate signature

Started by Dan Nicolaysen -   in API Authentication

My organisation wants to setup Xero instead of MYOB, but only if we can get the API to work. So far I have not been successful. The complex structure of our system prevents me from using any of the libraries and SDK. The connection to the API is done inside SQL server, and it currently works fine with others like MYOB, TomTom, FleetMatics, SMSTech and ABR. Since the Xero API is based on HTTP REST, it should be possible to make use of the API without any wrappers, but I can't find proper documentation on the actual calls to the API.

I'm getting 3 different errors depending on the mood of the API. Most of the I get "Failed to Validate Signature". I'm guessing it might be something wrong in the formatting of parameters. Other times I'm getting "(401) Unauthorised" and sometimes I'm also getting "Nonce has already been used". A bit strange, as all the calls are identical.

This is whats happening:

Public application and I have the right Consumer keys and Secrets. The API Previewer works.

1. URL to call is:

https://api.xero.com/oauth/RequestToken?oauth_consumer_key=(My consumer key)
&oauth_nonce=F270C70643C3265B2074C1B36A7B4FCB8A4552B4
&oauth_signature_method=HMAC-SHA1
&oauth_timestamp=1498625126
&oauth_version=1.0

2. URL is then URL Encoded.

3. The string "GET&"+URL is signed in HMAC-SHA1 and the key is the (My Consumer Secret) PLUS & (why the & at the end?)

4. The signature string is then URL Encoded and added to the URL as "&oauth_signature=(Signature)

5. Call is being made, and the errors comes.

Now, people might say "just use the wrappers", but then the API is not a RESTFUL API, but instead an API that can only be used via Wrappers. Not very smart. I would like to see a complete description of how to setup the right calls for making a Token Request. Right now, the information about the call I'm trying to make is only found by spending hours searching the Xero Community and previous posts. No where in the documentation is is saying anything about the actual signature and the content, how the signature must have "GET" or "POST" and the start, and how the secret key should have an "&" at the end. With 487 posts on issues with API Authentication I can see I'm not the only one.

Please give me a hint on how to get this working before I smash my workstation to smithereens in frustration.
Hey Dan,
My explanation will be split across two replies.

I'll try breakdown our OAuth 1.0a signing process for public apps .
I'm going to use getting a request token as an example.

The URL for getting a request token is https://api.xero.com/oauth/RequestToken and the method that should be used is POST
If not using a callback URL, wherever mentioned you must use the value oob


1. Create signature base string

The signature base string is created using the request method, request URL, request query parameters, and some extra OAuth parameters.
The extra OAuth parameters are:
oauth_callback, oauth_consumer_key, oauth_nonce, oauth_signature_method, oauth_timestamp, oauth_version

The signature base string is split into 3 parts:
{part 1}&{part 2}&{part 3}

The first part is just the request's HTTP method.
The second part is just the request's URL minus any query parameters, url encoded.

For our example that leaves us with:
POST&https%3A%2F%2Fapi.xero.com%2Foauth%2FRequestToken&{part 3}

Part 3 is created using any provided query string parameters along with the extra OAuth parameters mentioned earlier.
Using all of these parameters, you'll need to build up a list of key, value pairs, separated by ampersands, where the pairs are ordered lexicographically.
Ordered lexicographically basically means the keys are sorted alphabetically uppercase, then lowercase and if any keys are the same they are sorted by value. For example, if our keys were 'apple', 'banana', and 'Banana', they would be ordered 'Banana', 'apple', 'banana'.

For our example, the string would be:
oauth_callback={your callback url}&oauth_consumer_key={your consumer key}&oauth_nonce={a random nonce}&oauth_signature_method=HMAC-SHA1&oauth_timestamp={current timestamp}&oauth_version=1.0

We then need to URL encode this string so that it becomes something like this:
oauth_callback%3D{encoded callback url}%26oauth_consumer_key%3D{your consumer key}%26oauth_nonce%3D{encoded nonce}%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D{current timestamp}%26oauth_version%3D1.0

Using all three parts we now have our signature base string, ready to be signed:
POST&https%3A%2F%2Fapi.xero.com%2Foauth%2FRequestToken&oauth_callback%3D{encoded callback url}%26oauth_consumer_key%3D{your consumer key}%26oauth_nonce%3D{encoded nonce}%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D{current timestamp}%26oauth_version%3D1.0
 

Matthew Mortimer (Xero Staff)  

2. Sign the signature base string

As our example is a public app, we don't have a certificate to sign with and so must build up our own key.
The key is comprised of two parts:
{part 1}&{part 2}

The first part is just your consumer secret, (found in the developer portal along with your consumer key).

The second part is the key of the Request token or Access token you have depending on how far through the 'OAuth flow' you are.
- When acquiring a new Request token, this will be blank.
- When swapping your Request token for an Access token, this will be the Request token key of the request token you just authorised.
- When making API calls with an Access token, this will be the Access token key.

In our example, we are acquiring a new Request token and so part 2 will be left blank.
Our key will be:
{your consumer secret}&

Using this key, you then HMAC-SHA1 sign your signature base string, and then base 64 encode the resulting byte array into a string.
This signature is your oauth_signature used in the next step.


3. Create Authorization header

Using the oauth_signature you just created, and all of the other OAuth parameters mentioned in step 1, you can now create the Authorization header to be appended to your request.

The structure of the Authorization header is as follows (using the same oauth parameter values used in step 1, as they are used to validate your signature):
OAuth oauth_callback="{your callback url}", oauth_consumer_key="{your consumer key}", oauth_nonce="{your random nonce}", oauth_signature="{your oauth_signature}", oauth_signature_method="HMAC-SHA1", oauth_timestamp="{your oauth_timestamp}", oauth_version="1.0"


4. Make the request

Your final request should look something like this

POST https://api.xero.com/oauth/RequestToken HTTP/1.1
Authorization: {Your Authorization header}
...


I hope this helps you with building your integration.

Cheers,
Matt
 

Matthew Mortimer (Xero Staff)  

WOW. Thanks Matthew.

This explanation should be part of the "Getting Started" guide for the API. Very clear and easy to understand. I will give it a go asap. I will post the outcome of the test when done.

Thanks again

Dan
 

Dan Nicolaysen  

Sorry to say, but I can't get it to work. The description given above by Matthew is excellent, but for some reason I cannot get this to work on Xero. We are making API calls directly from the SQL Server with table-value functions returning the data from the API as a table. We can even use the table-value functions on joins with internal tables. It works great for MYOB API.

Since we are already calling many other APIs directly from the SQL server (including MYOB), its not related to missing features in order to execute Http Rest calls. I think it's related to the signature signing using HMAC-SHA1, but it works fine on another APIs. Anyway, I have to leave this and move onto other stuff, so we will not convert from MYOB to Xero right now. Maybe later when I have more time to play with it.

I would like to suggest an endpoint URL that can explain in details why the call is not working. So instead of calling the real Request Token URL and just get an error, developers can swap to another endpoint URL like https://api.xero.com/oauth/RequestTokenTEST where the output is a report on all parameters received and how the call is structured. Then it would be easy to spot where the problem is located.
 

Dan Nicolaysen  

I'm really happy to see this. I too have had endless and fruitless attempts to get a Private app working without using a framework. I agree with every word that Dan writes about the poor documentation and poor support that you have offered, and the useless error messages that your system responds with. Like Dan, I too have given up trying - I have better things to do.
 

Mick Moignard  

Hi Mick,

I'm sorry to hear that.

Have you taken a look through any of the example SDKs we provide in our dev portal?
Even if you choose not to use one of them for your integration they can still provide valuable information into the signing structure of oauth 1.0a.
 

Matthew Mortimer (Xero Staff)  

Of course I have. I've also had long conversations with your support staff, who have been helpful up to a point.

But it doesn't work and nobody at your end can tell me why in enough detail for me to solve the problem. I've validated my signature string elsewhere, so I'm down to your end coming up with a different signature string to that which I generate from the same parameters. But nobody at your end is able to tell me what signature string you generate from the request I'm making Nor, from the latest round of questions, could anyone tell me what length signature string you expect from me. Hugely frustrating, in exactly the same way that Dan noted on your forums. It would be nice if you had more complete documentation, and it would be nice if you could be more forthcoming about why things aren't working when they don't.

I compare all this to using Mailchimp's api. That was real easy to do, Good and complete documentation available on their site and the support from them when things didn't work was excellent.


 

Mick Moignard  

Hi Matthew,

I am using private Application. But recently I have started facing the problem as
oauth_problem=signature_invalid&oauth_problem_advice=Failed to validate signature
can you please look into it.
 

Ritesh Temgire  

Matthew: any news on some updated doco for this? You listed above some details for the Public app for Dan, and it would be nice to see some for a Private app. I believe that I'm doing everything right, but I still get the failed to validate signature response. I'm not entirely convinced that your signature validation method actually follows the Oauth guidelines, especially in the light of this responsehttps://community.xero.com/developer/discussion/47189645/ where he had to reverse his signature string partway through the process.
 

Mick Moignard  

I'm not familiar with the library/tech the developer in that conversation is using but you definitely do not need to reverse the signature before providing it as authorisation. It may have been that something in their own flow had been the reason they had to reverse it
 

Matthew Mortimer (Xero Staff)  

He was reversing it before B-64ing it. Are you also using the Windows Crypt stuff, and so also inadvertently reversing it and thus requiring us to do it? What do your wrappers do on Windows?
 

Mick Moignard  

I'm almost certain this isn't the case. We've used some of our SDKs on both windows, apple, and linux machines with no requirements to reverse the bytes
 

Matthew Mortimer (Xero Staff)  

So if he was using a crypt library build in to Windows..... ? Like Dan,I cannot get a nonwrapper app to authenticate, and I cannot see why, so I have to query exactly how you decode my calls and validate the signature. Which you don't document.
 

Mick Moignard  

I've done some quick research on this and it seems this class does in fact reverse the ordering of bytes after encryption and prior to decryption. I've not been a part of the dev team dealing with oauth for a while now so can't confirm that it's used in our RSA-SHA1 implementation but this could explain the reversal requirements. I'm not sure it would explain why other SDKs like NodeJS, Elixir, or Ruby for example don't require explicit reversals of byte arrays, unless the reversal is also hidden in the crypto libraries they use.

Can you please raise this with our support team so that the dev team responsible for our oauth implementations can look into this more closely?
 

Matthew Mortimer (Xero Staff)  

Have done, tho I'm mystified why you can't, given that you're part of Xero.
 

Mick Moignard  

It will be easier for our support staff to keep you in the loop with issues like this as the dev team in charge of our oauth implementations investigate, rather than myself becoming a messenger in the middle.
 

Matthew Mortimer (Xero Staff)  

Hi Matthew,
Thanks for putting together those first 2 posts, I just stumbled across this thread, can you add them to the documentation somewhere so it can benefit others?

Without that post I have no idea how long it would have taken me to solve the issues I was having, so how ever long it took you to write it, I'm sure it's saved me more than that.

Not all heroes wear capes.
 

Robert Cassidy  

Matthew: anything new here? I raised a ticket as you requested. I've spoken with James Coleman here in the UK and had an email conversation wih him, but he's also gone off the air and won'r respond to me either, and the support desk has been silent since August on this issue too. Are you at all interested in supporting people properly? All I want to know is how your stuff actually works, so that I can work with it. As you agree above, some stuff does require reversals and so on which is not part of the OAuth spec that you point us at. If only you would tell me how your authentication ACTUALLY works, I could probably get my app working.
 

Mick Moignard  

Hi Mick,

In case you're interested: we looked into the question of reversing the byte order for RSA-SHA1 (private and partner apps) and summarised our findings over on the relevant community post https://community.xero.com/developer/discussion/47189645/.

Regards,
Amy
 

Amy Martin (Xero Staff)