Getting 500 Internal Server Error while running EmrEtlRunner

For engineers AWS batch pipeline (Legacy)
1

Hello There,

I am getting RestClient::InternalServerError (500 Internal Server Error):

Thanks
Brijesh Singh

2

Hi Brijesh - it looks like your error didn’t paste into Discourse. Could you try editing your post to add it?

3 
 x elasticbeanstalk-ap-southeast-1-6767232329389/resources/environments/logs/publish/e-u8wejkw989/i-g787wjhe/_var_log_tomcat8_rotated_localhost_access_log.txt1461031264.gz
      +-> saltside-de-snowplow-maket-name/logs/raw/processing/var_log_tomcat8_rotated_localhost_access_log.2016-04-19-03.ap-southeast-1.i-g787wjhe.txt.gz
      +-> saltside-de-snowplow-maket-name/logs/raw/processing/var_log_tomcat8_rotated_localhost_access_log.2016-04-19-04.ap-southeast-1.i-g787wjhe.txt.gz
      x elasticbeanstalk-ap-southeast-1-6767232329389/resources/environments/logs/publish/e-u8wejkw989/i-g787wjhe/_var_log_tomcat8_rotated_localhost_access_log.txt1461034867.gz
      x elasticbeanstalk-ap-southeast-1-6767232329389/resources/environments/logs/publish/e-u8wejkw989/i-g787wjhe/_var_log_tomcat8_rotated_localhost_access_log.txt1461038469.gz
F, [2016-04-19T05:02:54.128000 #22689] FATAL -- : 

RestClient::InternalServerError (500 Internal Server Error):
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient/abstract_response.rb:74:in `return!'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient/request.rb:495:in `process_result'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient/request.rb:421:in `transmit'
    /tmp/jruby2269699594919101292extract/jruby-stdlib-1.7.20.1.jar!/META-INF/jruby.home/lib/ruby/1.9/net/http.rb:746:in `start'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient/request.rb:413:in `transmit'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in `execute'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in `execute'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/rest-client-1.8.0/lib/restclient.rb:69:in `post'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/elasticity-6.0.5/lib/elasticity/aws_session.rb:29:in `submit'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/elasticity-6.0.5/lib/elasticity/emr.rb:302:in `run_job_flow'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/elasticity-6.0.5/lib/elasticity/job_flow.rb:141:in `run'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/emr-etl-runner/lib/snowplow-emr-etl-runner/emr_job.rb:400:in `run'
    /home/ec2-user/market/maket-name/lib/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/method_reference.rb:46:in `send_to'
4

Hi @birju1100,

Were you able to run EmrEtlRunner successfully before? If so, can you think of anything you might have changed that could have lead to this problem?

Also could you paste your configuration file (with removed/masked sensitive data)? It looks like a permissions issue…

Regards,
Ihor

5

I just re ran and it got resolved.
Thanks