Snowplow-mini local Vagrant

I’d like to do some testing and exploration of Snowplow using Snowplow Mini on my local development machine rather than spinning up AWS infrastructure.

When I tried to setup locally with Vagrant using the following steps, the Snowplow Mini UI was not available as described here:

Steps to reproduce:

https://github.com/snowplow/snowplow-mini.git
cd snowplow-mini
vagrant up

Log output here:

vagrant port
    22 (guest) => 2222 (host)
 10000 (guest) => 10000 (host)
    80 (guest) => 2000 (host)
  3000 (guest) => 3000 (host)
  4171 (guest) => 4171 (host)
  5601 (guest) => 5601 (host)
  8080 (guest) => 8080 (host)
  8081 (guest) => 8081 (host)
  9200 (guest) => 9200 (host)

Host machine is a MacBook Pro running Mojave 10.14.5.

Hi @wadejensen ,

It is clear from the log output that build process has failed to complete. We’re aware that Snowplow Mini’s build process has a few fragile components that can easily break and we’ve started to address them. The next release is expected to bring build process to a durable state. You can see the ongoing 0.6.1 release work at https://github.com/snowplow/snowplow-mini/pull/197 .

In the meantime, you can use v0.6.1 branch and build locally fine. We aren’t done but it can be used for local test purposes. Please keep in mind that v0.6.1 branch will be updated regularly and can be in an invalid state for short periods until the release happens.

Please let us know how it goes.

Thanks,
Oguzhan

1 Like

Hi @oguzhanunlu thanks very much for getting back to me within the day :smile:.

I will use the 0.6.1 branch and jump around different commits until I find one that builds smoothly.
Cheers,
Wade

1 Like

I did a git bisect and I couldn’t find a commit that worked locally for me with Vagrant, including the 0.3.0 release in which Vagrant support was announced.

I think it is safe to say something is wrong with my Vagrant install as well. Will investigate and update on the outcome.

Hi @wadejensen ,

Thanks for sharing your progress. I don’t think it is about vagrant itself right now but the way Mini installs specific dependencies, which we’re updating within the 0.6.1 release.

You don’t need to jump over commits to find the one that works, if you’ve checked out the release branch v0.6.1 , running vagrant destroy && vagrant up command at the root of the repository should work fine, I just checked personally on my localhost & it builds fine. Note that there are recent force-pushes in the branch, so refresh your local branch before running a vagrant command.

If this still doesn’t work, then it might be a good idea to check if you can really boot up a vagrant machine. A fast option could be running vagrant init ubuntu/xenial64 && vagrant up inside a temporary test directory and checking if it is fine.

Please keep sharing how it goes.

Oguzhan

Hi @oguzhanunlu,

So part of the issue was that Mojave changed the permissions required for terminals to access the disk, which causes problem with Vagrant NFS mounts.

I resolved that issue by using the default macOS terminal (rather than iTerm2), which triggers system privilege escalation dialog confirmations.

I’ve tested Vagrant with a blank Ubuntu image as you suggested, and it built successfully.

Wades-MBP:tmp wjensen$ vagrant ssh
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-150-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.

New release '18.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


vagrant@ubuntu-xenial:~$ 

I’m able to progress much further through the ansible playbook now, but am still running into issues. I think due to a chown failing and subsequent filesystem access being denied.

The new terminal output from vagrant up on the root of the snowplow-mini repo on the v0.6.1 branch:

Hi @wadejensen ,

You’re right that this is a permission issue as the error messages indicate. I also use iTerm2 and have provided full disk access. You can check the related iTerm2 documentation on how to do it.

Regards,
Oguzhan

Hi @oguzhanunlu,

I’ve granted full disk access and tested with both iTerm and the default Terminal.

I’m still getting the same failures. See gist.

If you’re confident its an issue specific to my machine, then I’ll close this and move on to using the AWS AMI instead.

Hi @wadejensen that error is normally to do with a conflict between NFS and certain file permissions - as a last try could you try commenting out the NFS lines in the Vagrantfile, destroying your current Vagrant up attempt and then trying again?

The lines in question: https://github.com/snowplow/snowplow-mini/blob/v0.6.1/Vagrantfile#L7-L9

1 Like

Commenting out those lines did it @josh! Thank you both for your patience and assistance!

2 Likes