Xero - beautiful accounting software

Xero Developer Help Center

Xero Developer Community

Community > Getting Started >

oAuth 2 Authentication in Back-end Application

Started by Richard Parry -   in Getting Started

Hello, For the past year we've been using oAuth 1.0 with Xero - this allows our back-end application to run every day to upload our sales invoices as well as a PDF of the sales invoice. It uses both the Contacts and Invoice "scopes" (or whatever they were called in oAuth 1.0). This works great, but now we need to upgrade to oAuth 2.0.

Our back-end application has no UI - It's simply a web script, built in ASP Classic (Yes, I know it's old, but it is what we know and works) that is called using a scheduled task.

I've reviewed all of the oAuth 2.0 documentation and we can integrate with this, however I am stuck with it requiring a call-back URI. The problem is, as this has no UI then it has no ability for the user to click the "Grant Access" button on a website, which the authorise URL requires.

I've had a look at the page here; https://developer.xero.com/documentation/api-guides/machine-2-machine
But this is no use, as it still seems to require running permanently and refreshing the token every 30 minutes. As mentioned above we only run our synchronisation script every day at 6pm. This help guide also uses what appears to be a Linux based xoauth programme, which is no good for us running on Windows Servers.

What solution do we have here please?
Hi Richard.

Last year we started migrating all our API traffic to our OAuth2.0 gateway which as you have called out require a user interaction to generate an access & refresh token to maintain long term access.

More can be read about the business decisions to do so here: https://devblog.xero.com/an-update-on-why-we-are-saying-goodbye-oauth-1-0a-hello-oauth-2-0-6a839230908f?gi=993655e9dc66

In short, it allowed our dev team to free up time from maintaining a more complicated API gateway with the 3 previous app types. However there were a lot of devs like yourself using the "private app" functionality where this change introduced an additional level of complexity.

The team is working to build a solution using OAuth2's client_credentials grant to better mimic the experience of developing on our previous private app type.
https://oauth.net/2/grant-types/client-credentials/

This, however, will be a paid service so we can better support provision support for it into the future. https://developer.xero.com/announcements/custom-integrations-are-coming/

If you want to continue using the API for free you can setup the OAuth2 access/refresh token logic, as supported by our tools like xoauth, etc.

Let me know if this clears up your path forward or creates more questions!
 

Christopher Knight (Xero Staff)  

So Christopher, you are basically ending support of the free API for machine to machine integration?
Do you have any guidance yet on how much you are going to charge for the service?
I understand that it is due 'early 2021' and that OAuth1.0a will be supported for 90 days after delivery so that users like Richard and myself can make the switch. Can you give any more detail on dates, please?
I'm not sure how the xoauth tool would work as a replacement for a completely hands off integration that currently works under OAuth1.0a. Can you explain how you see xoauth working in such a scenario?
 

Greg Shaw  

Hi Greg.

This FAQ is the only data I have at the moment: https://developer.xero.com/announcements/custom-integrations-are-coming/

I suspect we will be updating this very soon with the additional information such as cost and technical migration resources.

However the current OAuth2.0 authorization_code flow can be worked to support 'machine to machine' like API connections. Once the access_token is generated, either using XOAuth or simply a rest client, you can plug the token_set (access & refresh token) into your script and then you just need to refresh and replace the access_token prior to each script run.

https://developer.xero.com/documentation/oauth2/auth-flow#refresh:~:text=Refreshing%20access%20tokens

So to re-iterate, once you generate the initial token, the following logic should be embedded in your script to get a fresh access token each run.

POST https://identity.xero.com/connect/token
authorization: "Basic " + base64encode(client_id + ":" + client_secret)
Content-Type: application/x-www-form-urlencoded

grant_type=refresh_token
&refresh_token=xxxxxx

---

Here are two tutorials I made on the subject that might be helpful if you choose to go that route.
https://www.youtube.com/watch?v=Zcf_64yreVI
https://www.youtube.com/watch?v=H_k8Z8Zq99s
 

Christopher Knight (Xero Staff)  

Richard, this may not be for you but I managed to overcome this by using an Azure function to regenerate the access token every 25 minutes (using the refresh token) and storing it in Azure KeyVault. It's only been running for around 18 hours but I've come in to work this morning and tested the latest bearer/access token and it's working. It was only about 100 lines of code in the end.

The only problem I see is that if there is a change to scope, change to organisation selection then I'll need to regenerate the refresh AND access token manually through postman and update it manually into KeyVault. I'm not hitting it with a big stick to begin with, will pick off pieces of functionality as I go which will require scope changes, I'm sure.
 

Brad Dixon  

Thanks for the replies, but I am still non-the-wiser on what I should be doing. Should I be trying to figure out XOAuth and spending tons of time on that or doing something different? This whole process seems to be overly complex and I dont understand why.
 

Richard Parry  

This is really doing my fruit in now.

I've tried so hard to find how this XOAuth works - The only documentation I can find is the animation but it goes so quickly it's really hard to follow.

I've run this but after running the connector command and authorising the application in the browser it returns the following;

? Choose a client DataSync
Requesting OIDC metadata from https://identity.xero.com/.well-known/openid-configuration
Received OIDC metadata for authority: https://identity.xero.com
Opening browser window
Received OIDC response
Exchanging code at token endpoint: https://identity.xero.com/connect/token
Validating token
Using public key: XXXX (Removed for security)
Token is not valid yet

The browser also says "Token is not valid yet".

I dont get it - I really do not understand why someone intentionally made this so complex and dificult to use. All I want to is mass-upload some invoices on a daily basis - Something that was easy peasy with oAuth 1.0, now it's become almost impossibly complex it's making us consider moving our entire company accounts to another provider.

Even if I do manage to get XOAuth working, do I need to be reminding myself to run this every 60 days or something? Again, absolutely no documentation or information on how this works or even why it has to be done.

How can we get this sorted out?
 

Richard Parry  

After digging around I found this post;

https://developer.xero.com/announcements/custom-integrations-are-coming/

It says early 2021 private application integration is coming, however it says nothing about when?

We've got a deadline, imposed by Xero to complete this change-over by end of March - That's just over a month away and yet no idea about this integration for a premium oAuth 2.0 system?
 

Richard Parry  

Hi Richard, whilst I'm developing systems in PHP nowadays, I have VBscript knowledge and you might want to pick up what Christoher was commenting on

"POST https://identity.xero.com/connect/token
authorization: "Basic " + base64encode(client_id + ":" + client_secret)
Content-Type: application/x-www-form-urlencoded"

by running a Winhttprequest as a scheduled task (run it in a .vbs file and it should execute ok on your machine. This is a VBscript equivalent of cURL and the parameters can be injected into the refresh request and the task can run every 15-20 mins or so if you are on a 30 min timeout , or longer if you are on a 1 day session timeout

SO link for more info > https://stackoverflow.com/questions/37462580/how-can-i-post-data-using-curl-in-asp-classic/37462944

Good luck!

Mark
 

Mark Lawson  

Thanks Mark - That is a great reply - I actually already use exactly that mechanism for running lots of scheduled tasks like this, and its exactly how I get the current Xero script to run in the background.

I've tried the above, but I get an "invalid_client" response, however if I have read the docs properly then this is because i've still not obtained the authorisation code from the initial call-back done in the browser. This is the exact bit I am struggling with due to it being a background task.

I understand this authorisation code can be obtained through XOAuth, but then it brings me on to how to get this tool working, as at the moment per my earlier reply today I couldn't get XOAuth to work and even if I did, would I haveto run it manually every 60 days or less?
 

Richard Parry  

Hey all.

Richard - For context we moved away from the private apps / long lived access token for security purposes to the industry standard of OAuth2.0. We currently only support the "authorization_code" type grant, which requires human interaction (once) to generate an initial access_token / refresh_token pair. This can be infinitely refreshed using this code: https://developer.xero.com/documentation/oauth2/auth-flow#refresh:~:text=Refreshing%20access%20tokens

XOauth is simply a tool to make the initial token generation easier for folks like yourself. You can also simply use a rest client to generate that initial access/refresh token combo: https://github.com/XeroAPI/Xero-Insomnia

That said, you will still need to automate the refreshing of the expired access_token in your script if you go with the authorization_grant option.

-----

If you are currently using private apps the March 30th deadline is being extended. Private apps will continue to be supported until 90 days after the production release of the "Custom integrations" feature. Under the hood this will be the OAuth2.0 "client_credentials" grant.

https://tools.ietf.org/html/rfc6749#section-1.3.4

It will still require you to POST to our server to get a 30 minute access_token, but the parameters will be gotten from your /myapps dashboard, and we will not require a human to login.

--------

Sorry about your fruit :) Can keep fielding your questions here if you have them.
 

Christopher Knight (Xero Staff)  

What he said.... less flippantly, you'll need to use the web UI initially to create an access token instance and then run your automated task with the initially issued access token using the grant_type=refresh_token parameter. I'll give it a go myself (although my flavour is Linux, I'll try it on my local PC as a scheduled task)

edit: assumption is that the refreshed AT can be recycled/reinstantiated in perpetuity..
 

Mark Lawson  

Why don't you integrate the initial token generation into Xero UI? At least this will simplify life for most of developers.
 

Roman Strashkin  

Just giving an update on this. After spending several days on this I finally got it working.

I am sure there's a good reason for the complexity, but I've got plenty of other API's integrated with other systems using oAuth 2.0 and none of them are this complex.

For Xero, you've got to submit an online authentication to give you an access token and refresh token. Then you've got to refresh that every 15 or so minutes to ensure it doesn't time out, otherwise you'd have to generate a new access token from the start again, which is the manual process. The xoauth in Windows doesn't work for me - just has an error saying token not ready every time I try to do the connect phase.

The other thing I struggled with was the Base64Encode - For some reason Xero require strict Base64Encode, and there's actually 2 ways. Fortunately we use Chilkat software for various functions which aren't native to ASP, so this has a Base64 Encode feature that worked.
 

Richard Parry  

We've developed a "proxy" endpoint that manages all of this for you with a much more backend-friendly auth, renewing the tokens, resolving and retrying any HTTP errors (Rate limited requests, server downtime etc.), happy to discuss redistributing it over email t.dale@cross-solutions.com.au
 

Taylor Dale