Schemas not available on iglu-server/schemas/public

1

Hello everyone,

I have set up a local Iglu server and pushed about 50 custom schemas using igluctl with --public flag. However, when calling the Iglu server’s API, it tells me {“message”:“No schemas available”}. I checked the Postgres iglu_schemas table and all rows have is_public: true.
The request I’m sending is
curl -X GET “http://ec2-xx-xx-xx-xx.us-west-1.compute.amazonaws.com:8080/api/schemas/public” -H “accept: application/json”

What I should maybe mention is that I first pushed the schemas without the --public flag, and then deleted all the private schemas from the DB so that I could re-add them in public via igluctl.
The same schemas worked in Snowplow mini.
Any ideas how I can fix this? Any help is appreciated!

2

I am wondering if this is some kind of odd caching issue. We have seen something similar in snowplow mini.

1 Like
3

Try and restart your iglu server, I’m not sure exactly what the problem might be but I suspect manually deleting the entries from the DB might have caused the issue. Generally I’d advise don’t delete schemas from an Iglu, simply upload new versions on top of the old ones.

1 Like
4

I reset the database and re-ran the iglu-server setup, still getting {“message”:“No schemas available”}.

Concretely, I did the following steps:

  1. connect to postgres, drop igludb database, create new igludb database
  2. java -jar iglu-server-0.6.0.jar setup --config iglu.conf
    concretely, my config is:
repo-server {
  interface = "0.0.0.0"
  port = 8080
  pool = "cached"
}
# 'postgres' contains configuration options for the postgre instance the server is using
# 'dummy' is in-memory only storage
database {
  type = "postgres"
  host = "localhost"
  port = 5432
  dbname = "igludb"
  username = "postgres"
  password = "xxxxx"
  driver = "org.postgresql.Driver"
  maxPoolSize = 5
  pool = {
  }
}
# Enable additional debug endpoint to output all internal state
debug = true

  1. connect to igludb database,
    INSERT INTO iglu_permissions VALUES ('xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx', '', TRUE, 'CREATE_VENDOR'::schema_action, '{"CREATE", "DELETE"}'::key_action[]);

  2. run iglu-server
    java -jar iglu-server-0.6.0.jar --config iglu.conf

  3. push schemas to iglu-server

./igluctl static push ./schemas ec2-xx-xx-xx-xx.us-west-1.compute.amazonaws.com:8080 xxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --public
TOTAL: 53 Schemas successfully uploaded (53 created; 0 updated)
TOTAL: 0 failed Schema uploads
  1. try to access schemas via API fails
curl -X GET "http:// ec2-xx-xx-xx-xx.us-west-1.compute.amazonaws.com:8080/api/schemas/public" -H "accept: application/json"
{"message":"No schemas available"}

Am I missing some step here?

5

I believe you need to specify your API key that has the permissions to retrieve the schemas on the GET request:

-H "apikey: YOUR_APIKEY"

See here for more info: GitHub - snowplow/iglu: Iglu is a machine-readable, open-source schema repository for JSON Schema from the team at Snowplow

1 Like
6

Hey @boba,

To add what Paul said, there was an API change in 0.5.0 and we deleted /api/schemas/public endpoint. Now, all schemas are served via /api/schemas and only apikey header determines what schemas will be returned. If you’re anonymous user (no apikey provided) - you’ll get only public schemas (as with /api/schemas/public in pre-0.5.0). If you provide an apikey - it will return you schemas that your key has access to and public ones.

2 Likes
8

Ooh I see, I called api/schemas and it works! Maybe we should change the response of api/schemas/public to {"message":"deprecated"} instead of {"message":"No schemas available"}

Can I specify an API key for the StreamEnrich Iglu client?

9

Yes - you can specify this as part of the resolver configuration that is passed to Stream Enrich.

1 Like