API Key Authentication

[kgarner7]

Derived from https://gist.github.com/kgarner7/33a6ab836528837b7ece44409ceb2d95, with numerous suggestions from @deluan and @jeffvli and discussion in Navidrome Discord.

In short:

1. Proposes redefining the meaning of p to be API Key, instead of password. Provides guidance for servers on providing these keys, as well as one new endpoint for client benefits for documentation.
2. Recommends deprecating token/salt-based auth.
3. Recommends a new endpoint/authentication scheme, Media API key, which uses temporary tokens generated by the server. While I believe this would be a benefit for both clients and servers (especially for things like rpc), I'd be fine backing out this change.

Ultimately, this proposal isn't anything revolutionary (like using OAUTH2, refresh tokens/etc).
The primary goal is to provide a way for clients/servers to authenticate without either party having to store the user's passwords in a reversible fashion.
By repurposing p for this, clients require no functional changes (although some UX to note API key and point to URL), and server changes should hopefully be minimal.

I also recognize other parts of the page should probably be updated with the new auth scheme (if accepted). This is a draft, after all.

[netlify]

✅ Deploy Preview for opensubsonic ready!

Name	Link
🔨 Latest commit	4b94bc6
🔍 Latest deploy log	https://app.netlify.com/sites/opensubsonic/deploys/6713059bcbc341000889b0dd
😎 Deploy Preview	https://deploy-preview-99--opensubsonic.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[Tolriq]

Why do all the discussion outside of this forum where all the involved parties can interact?

Anyway the repurpose of p to be an API key is 100% agnostic to the clients, supporting an header that also requires the username as a query param does not really makes sense from a security pov on the contrary.
So there's no need for an API extension to expose this and servers can already do that.

An extension would be needed if we replaced the auth for an API token that does change the API surface and that contains the information about the user and the perms.

That also brings the question of the getopensubsonicextensions that we may need to make non authenticated so clients can know about new auth support. (But with the risk of exposing more attack surfaces with anyone knowing about all new endpoints, small as they can scan them anyway).

The other issue is the media key, while the need is real, it's a breaking change, as to use them we would need to not pass the actual auth to the endpoints. (And support should be for all endpoints, users can use stream to download, and will require the lyrics and ... should not lead to random results depending on the server).
And IMO they should be tied to specific media and not a generic key, 90% of the servers use incremental number ids so once you have 1 url with the key you have access to everything.

And as a reminder: OS API must not have should on API endpoint support or parameters or anything API surface related. It's either must, or there's an API endpoint that allows for clients to list what is supported and what is not. Let's keep the main purpose of OS: Clients must be able to have something deterministic when they use the OS API.

[kgarner7]

1. The last time the auth issue was updated is multiple months ago. Given the considerable inactivity (and the discussions being quiet as a whole) I wanted to work in a forum where I could get rapid feedback and get something out to force the issue again.
   2 Redefining p (and deprecating salt/token auth) is at the very minimum a clarification.
   Especially with an endpoint (keyUrl) to get metadata about the keys, it is a change clients would need to know. Even if this is the only thing of this that survives, I would still insist that this should be an extension. Specifically because there would be need for client UX to reflect that this is an API key, not a password. Some servers may already do this, while others may not which already leads to client differences. Explicitly forcing servers to have separate client authentication from password is the ultimate goal. (I guess technically there isn't a reason to deprecate salt auth either, as that can also be API key)
2. I would personally love for the token and username to both be part of header (or just require token and no username). Dropped token in header and username in query is also fine
3. Are you saying media keys (or something similar) would need to be item-specific? I don't know how it is breaking for clients as-is, because it's a new parameter. I understand the "should"s being annoying there, and I would also be fine just having it for just coverArt and stream. Or not at all and dropping this portion entirely.
4. As for the other should (around scoping), this is because I have very little faith it would be possible to agree on a set of api endpoints for various scopes. I would love to codify a set of endpoints per different scopes and return that (and allowed music folders) in some endpoint which validates a token, but I don't want to discount servers having their own ability to scope and was worried it would be too complicated server-side.
5. I suppose related, but yes, I would say that getOpensubsonicExtensions should be unauthenticated, as that is the only way a client would be able to know what authentication the server supports. At least, it would be necessary with this extension as it is specifically related to authentication unless clients have to rely on checking for the presence of a generic keyUrl endpoint.

[Tolriq]

The whole problem is that if nothing is properly codified then it's the return of the mess on the client side.

Refactor of p:

1. The keyURL does not bring anything or anyknowledge to the client, it's just an url for the user to read things, the client can't deduce anything, specially if you want that call to be not authenticated, the client does not know if the p is a password or an apikey for that user, and in all cases it absolutely does not matter for the client.
2. If the server want's to enforce apiKey, then the proper solution is to add a new error code when the user tries to use the API with it's password, error 42 for example with the message being that url so the client is aware and can properly know when to redirect the user to it. (Or eventually add a new field helpUrl to the error messages, to support both a specific error message and an url for the user).
3. If the API key can have specific permissions that are different from the normal Subsonic behavior, then it should be codified and a new endpoint must be added so the client can know what it can call. Having calls erroring very deep in client codes without even a way to know that it's because the API key is restricted is not a proper way for clients to deal with and warn users. Clients having a call to know what they can do and warn the user before doing things that the API key is restricted is the proper way. (Specially if you want the API key to limit to some music directory too)

TL;DR; there's no need for an apiExtension to refactor p to be an API key if the server wants, just a new auth error. Even older clients, will see an error and show an error to the user making them go to the server GUI to check.

If you want to do more then a proper global solution is needed, new param apiKey that replace u/p/t ..., endpoint to know what the apiKey have access to, and eventually other information like a duration, limited to a media, ...

mediaKeys:
Imo they need to be per media for security purpose. A key might be requested for a long duration and give access to everything via the known ids.

And if you want them you need to support them in all the endpoints related to playing those media including download, lyrics and everything. Why would an user be able to play a media but not access the lyrics when using those when it would work with another key.

And if we add support for apiKey for the auth then we can reuse it for the media access by generating temporary apiKey limited to 1 media. That simplify things and is in line with a proper simple auth scoped API.

But to resume, the very most important part will always be: No should and if there's scoping then the scoping needs to be codified for clients to know and warn the user properly. We do not want to return to random behavior between server with what we add, this break the most important contract of OS.

TL;DR:

Choice 1) allow server to enforce apiKeys as P by adding a new error code and an url for the user to create / manage them.
Choice 2) really implement scoped apiKeys that replace auth, have an end point to know the associated permissions and details.
Choice 3) same as 2, but add an endpoint to request temporary apiKeys scoped to readonly for a single media.

Remaining question about "mediaKeys" what about scrobling ?

Personal opinion, since oauth2 would probably be too much work for most servers / clients anyway, I think apiKeys as already suggested in the other threads is an acceptable auth system for apps and so think 2 or 3 are a nice solution as long term permission improvement and would solve a lot of current issues.

[epoupon]

By repurposing p and renaming authentication labels on clients, does this mean we have to officially drop forward authentications using LDAP, PAM, etc.?

[Tolriq]

No, such repurpose as explained in point 1 would be server side only with a new error code and fully optional as it would not change anything at all from client side.
Client could optionally support the new error code to update it's interface to say it requires an API key.

Forcing changes to p is a breaking change and not compatible with OS.

[epoupon]

I am not sure to understand. With a given p value, how does the server can know it is an API key or a regular password to instruct the client via an error code?

[Tolriq]

Well the obvious way is that if the server want to force apiKeys in p it check if the key exist and return the error?
It's internal to how the server want to deal with this if we choose this simple change that have 0 impacts on clients.

[paulijar]

Just to add my 5 cents, using the p argument for a generated API key is what the Subsonic backend on the Nextcloud Music has been doing from the day one, so that's definitely already possible as @Tolriq has been telling. We have never supported passing the actual user password to the Subsonic API. If the given p is not a valid API key, then we just give the error code 40 with message "Invalid login". Having a dedicated error code in case the p is actually the password of the user could of course improve the UX on the client.

What would actually improve the security, would be to have the concept of user session in the Subsonic API with a temporary session token. This way, the actual permanent API key (or password if you will) would be used only on one handshake message and most of the API communication would happen using a temporary token obtained as a result of the handshake. This would fix (or at least improve the situation with) what I see as the main problem of the current system: Quite often the users post their log files online with their bug reports, without detracting the p from the list of the URL arguments. I suspect that this happens more easily in case the client uses the enc: format to pass the p argument because that makes it less obvious that the password is there in fully readable format.

[dweymouth]

One of the problems with expiring tokens that has been discussed is clients often rely on semi-permanent URLs for streaming tracks (and sometimes fetching images) - for example if the user has loaded up a play queue on a client, and they pause the app for weeks or months at a time, and then return to it, the playback URLs for the tracks in the queue should still be valid (unless explicitly revoked by the user) or else playback may fail.

[dweymouth]

Just to add my 5 cents, using the p argument for a generated API key is what the Subsonic backend on the Nextcloud Music has been doing from the day one, so that's definitely already possible as @Tolriq has been telling.

I would actually make the argument that that is not compatible with the current spec, since the current spec implies all logins are password-based, so clients will ask users for their "password" not "API key", which could potentially lead to a confusing UX. I suppose if Nextcloud music makes it very clear in the docs or when creating a key that it should be used instead of password for all Subsonic clients, that may clarify things a bit, but it would still be nice if the spec is designed such that the client knows when to ask for an "API key" vs "password"

[paulijar]

for example if the user has loaded up a play queue on a client, and they pause the app for weeks or months at a time, and then return to it, the playback URLs for the tracks in the queue should still be valid (unless explicitly revoked by the user) or else playback may fail.

Why would the client store the playback URL of all the tracks on the queue? The URL may be formed based on the song ID the moment the playback starts (or is resumed). If then playing the URL (or any other API call) would fail with "session expired", then the client would need to start a new session and retry the operation. Of course that would require adding a small bit of extra logic but there are no free lunches.

[dweymouth]

for example if the user has loaded up a play queue on a client, and they pause the app for weeks or months at a time, and then return to it, the playback URLs for the tracks in the queue should still be valid (unless explicitly revoked by the user) or else playback may fail.

Why would the client store the playback URL of all the tracks on the queue? The URL may be formed based on the song ID the moment the playback starts (or is resumed). If then playing the URL (or any other API call) would fail with "session expired", then the client would need to start a new session and retry the operation. Of course that would require adding a small bit of extra logic but there are no free lunches.

Many (most?) clients don't handle streaming the audio data themselves but pass the stream URL off to a 3rd party framework for playback (libmpv, etc). Clients that play gaplessly must at a minimum have the next-up URL loaded by the playback library ahead of time, alongside the current one. So if a users pauses a play queue for a long time, both the current and next playback URLs should remain valid in order to avoid playback errors. Also, with 3rd party playback engines it may not be possible to detect and handle a specific auth error differently than a generic "playback failed for some reason" error

[lachlan-00]

I think that mediaKey should be added as an extension of the user instead of a separated into an endpoint.

• The user in Ampache has:
  • password
  • ApiKey (used for password auth in Subsonic)
  • StreamToken (Ampache version of mediaKey)

The streaming token can only stream media and all streaming URLs are generated using this token when it exists. Adding it to the user responses allows a client to build their own URL's with the token

• If i extended this to Subsonic I don't need to make any changes to the Subsonic API outside of:
  • Add the StreamToken to user responses.
  • Allow user access to stream, download and scrobble methods using this key
  • Allow user reset of this mediaKey in updateUser or separate method

This would all be transparent to older clients without the need for new endpoints.

In regards to removing / deprecating / changing auth parameters a new error message to say "deprecated/removed" and then as a server I can allow or deny access to this auth method with a config option on the server.

A blocked by default approach means people can still open their servers up and we can push people towards different methods.

To extend though; I still think the simplest path is to move these parameters out of the URL and into headers, Ampache uses a Bearer token and supports all types of auth. This would allow a server to block auth from the URL (optionally) and require header auth instead.

If you recommend header auth, implement a streaming mediaKey and allow blocking auth-related URL parameters on the server you've solved most of the problems of having reversible / plaintext auth in the URL.

[Tolriq]

Quite often the users post their log files online with their bug reports, without detracting the p from the list of the URL arguments.

This is the server job to hide those usually, most tools either directly hide them or have a download redacted logs button.

I would actually make the argument that that is not compatible with the current spec, since the current spec implies all logins are password-based, so clients will ask users for their "password" not "API key", which could potentially lead to a confusing UX

This is not breaking in sense of API, from API pov a password or an API key does not change anything. With a new error we improve user feedback. (But in all cases I still think that a new auth with apiKey that replace u + p is better globally)

I still think the simplest path is to move these parameters out of the URL and into headers,

This was explained a lot of time already that this is not possible due to casting and some renderer limitations, it is not possible to have upnp devices send headers for example.

If we start to mix a new enforced auth via headers and another one via keys for some endpoints, this becomes really messy and we should directly rewrite new endpoints.

[Tolriq]

So @kgarner7 what do you prefer?

• Just the error code to allow quick solution server side only.
• New apiKey auth system that replace current system (So without different permission level handlingà
• New apiKey auth system with support for scoping (Access points, library, media)

[kgarner7]

So @kgarner7 what do you prefer?

* Just the error code to allow quick solution server side only.

* New apiKey auth system that replace current system (So without different permission level handlingà

* New apiKey auth system with support for scoping (Access points, library, media)

Sorry for the delay. I think the best (and simplest) way to move forward would be starting with option 2, an api key auth (just API key in header, or k/something similar for query parameter). I would love to have expiring tokens/scopes, but that might be best as a v2?

[Tolriq]

Ok so :

Quick proposal to actually write :) It's a quick global view as busy right now.

Generic:

• extend error message with an optional helpUrl field that can help for a few cases.

1. Version 1

• new error code 42: Password authentification disabled, use api keys.
• getOpenSubsonicExtensions no more requires auth to access (But needs proper docs as the general doc says auth is always mandatory)
• new extension apikey version 1
• new query param k=XXXX for auth everywhere. (Name TBD maybe token or apiKey to be more explicit, no need to be a single char)

2. Version 2 (extension to V1) (so extension apikey version 2)

• All from version 1 +
• new error code 51: apiKey does not have access to this resource
• new error code 52: key validity duration requested exceed maximum allowed value
• new endpoint getApiKeyPermissions -> describe the scope and perm of that key TBD with a proper schema
• new endpoint requestApiKeyForMedia (param: songId + duration) return key + expiry or error 52 or 70 (returns an api that have access to everything related to the media, stream, download if the server allows download, lyrics, ... in all cases clients can use the stream endpoint to download.

If that works for you and have time to actually build the PR.

I think this is more in line with the spirit of OS in regard to API definition, retro compatibility, predictability and everything

[kgarner7]

Since I suspect scopes (if they even make it in) would take quite a while, I'll probably advocate for something more like:

1. V1 (as is)
2. V2 (or also v1) temporary token for explicitly denoted endpoints
3. V3 (if it happens) scope.

Either way, I'll give a go updating the PR. Thanks!

[Tolriq]

Unfortunately IMO in the way OS API is supposed to be build, to be able to have temporary token we need a way to know the limits of that token and so the scoping.
I do not think it makes sense to have 2 different end points for the scoping data specially if from a key we can't guess it's type and from the needs exposed I do not think there's a need to have different keyType with different parameters to achieve the same purpose.

BTW I forget but maybe also add an error 43 or whatever Invalid apiKey to differentiate from invalid login/password ?

[Tolriq]

Thanks.

@kgarner7 a few quick notes:

• If you want the keyUrl tied to the extension then the extension page doc must say that that endpoint is part of the extension and needed so that the extension page covers all the new endpoints.
• Same for mediaKey
• For mediaKey I still think that support for download and getlyrics is mandatory for clients to be able to really use all the media related functions.
• I would add an error 44 Invalid apiKey
• depending on how servers would store the mediaKey I think that an error 45 Expired apiKey could make sense, but if servers quickly purge the temporary keys then they won't be able to return it.
• ? new error code 51: apiKey does not have access to this resource
• ? new error code 52: key validity duration requested exceed maximum allowed value

All those are mostly details, but I think there's one thing that needs discussion / validation by more clients / servers:

Expiration, in seconds. Minimum value of 60, maximum of 86400 (1 day)

Any reason / arguments on those specific values?

[kgarner7]

...I actually meant to remove key url and other parts

[Tolriq]

Lol sorry some merge error then :) Will wait before commenting again :)

[kgarner7]

Nah, that's just my bad for not paying attention for commit. Anyway, I've removed any reference of the v2 temporary key stuff (and keyUrl is redundant with changes to error)

[Tolriq]

Ok so still the question about "an error 44 Invalid apiKey" to differentiate from the 40 invalid login/password. Else change the message / description of 40 to also cover apiKey ?

And api-reference still talk about passing the Authorization header. If this is wanted it should be documented in the extension too.
With a clear both must be supported as having the header optional would be hard to detect for the clients.

[Tolriq]

I think the new apiKey version no more pass an username so it can't be a wrong user name.

[kgarner7]

The reason why I left username is if it's not required (and clients just specify api key), then there's no easy way to get the username. I would potentially be amenable to adding a new endpoint to turn a token into a username

[Tolriq]

Ho you left username mandatory I missed that. Well then it makes sense but won't it be a problem if we extend to v2 with apiKey that can be limited to a media and don't want to leak the username in the urls ?

[kgarner7]

Sounds like a new endpoint to exchange token for username (and other things (?) for v2) then

[Tolriq]

Sounds good, could return the scope too ;)

I think better to reuse the apiKey for media than adding again something else that would not bring anything more.

[Tolriq]

double in in

[Tolriq]

Same as above.

[Tolriq]

Optional question but could there be a case in your needs for v2 where a key could not be tied to an user and we should define a default value? (Probably not but just checking)

[epoupon]

Bad copy/paste?

[kgarner7]

Optional question but could there be a case in your needs for v2 where a key could not be tied to an user and we should define a default value? (Probably not but just checking)

If we are allowing API keys to exist that are separate from users (e.g. system account, say for just Jukebox/images), then I think for something like that it would make sense for the username to be possibly null.

[Tolriq]

null is not valid, so if would be empty if not a well know value. For the if that's the question, I can't think of a reason but maybe some were evoked during the initial talk of this hence wanting to confirm.

If not, then let's merge this. In the worst case we can amend with a comment the empty state that would not change the API surface unlike a well know value that could be already used by a server as username and we"d need to protect somehow.