← Go back to NixNet Status

XMPP downtime

April 27, 2022 at 9:07 PM UTC

XMPP

Resolved after 175h 36m of downtime. May 5, 2022 at 4:44 AM UTC

I’m not entirely certain what’s causing the issue yet, but ejabberd (the XMPP server) is failing to start. I’ve pasted the relevant logs below; if you have any tips for getting it fixed, please let me know! (21:07 UTC — Apr 27)

After poking around quite a bit and with the help of some friends more familiar with erlang, this looks to be an issue with LXC. I’ve created a discussion in ejabberd’s GitHub repo to try and confirm before shifting my attention to debugging LXC. (04:51 UTC — Apr 28)

I took the issue to the LXC IRC channel and talked with one of the developers then we moved it to the forums to continue there. (14:02 UTC — Apr 29)


The Erlang/LXC issue is unfortunately still not resolved so I decided to grab another VPS and move ejabberd over to it. If your DNS caches expired earlier today, you probably noticed that your client suddenly logged you in and you were able to message people again! If you’re still not able to connect, you’ll just have to wait a bit longer; that part is out of my hands now.

Outgoing messages work fine. File uploads work fine. MUCs should work fine. Audio/video calls are a work-in-progress. Not entirely sure what the issue there is yet but I’ll work on it tomorrow. Incoming messages from your contacts likely won’t work for a little while longer. Again, this is down to DNS caches expiring. Once your contacts' servers pick up on the DNS change, they should be able to successfully message you again.

This took much longer than expected and I do apologise for that. My finances are a little tight at the moment and getting another VPS was my last resort. I’ll still work towards resolving the issue with Erlang/LXC because I would like to move XMPP back to my primary server as soon as possible though; this VPS is more expensive than I would like but I wanted to make sure it had enough capacity to comfortably fit everyone. If you’re able to donate to help support this new server, it would be much appreciated: nixnet.services/donate. If you’re interested, it’s the CPX31 in Hetzner’s Falkenstein datacenter. (04:44 UTC — May 5)

Amolith


2022-04-28 04:41:06.994538+00:00 [error] <0.181.0>@proc_lib:crash_report/4:525 CRASH REPORT:
  crasher:
    initial call: supervisor:eimp_sup/1
    pid: <0.181.0>
    registered_name: []
    exception error: no function clause matching lists:seq(1,unknown) (lists.erl, line 243)
      in function  eimp_sup:init/1 (eimp_sup.erl, line 45)
      in call from supervisor:init/1 (supervisor.erl, line 301)
      in call from gen_server:init_it/2 (gen_server.erl, line 417)
      in call from gen_server:init_it/6 (gen_server.erl, line 385)
    ancestors: [<0.180.0>]
    message_queue_len: 1
    messages: [{'EXIT',<0.180.0>,normal}]
    links: []
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 28
    reductions: 242
  neighbours:

2022-04-28 04:41:06.994512+00:00 [error] <0.179.0>@proc_lib:crash_report/4:525 CRASH REPORT:
  crasher:
    initial call: application_master:init/4
    pid: <0.179.0>
    registered_name: []
    exception exit: {{function_clause,
                         [{lists,seq,
                              [1,unknown],
                              [{file,"lists.erl"},{line,243}]},
                          {eimp_sup,init,1,[{file,"eimp_sup.erl"},{line,45}]},
                          {supervisor,init,1,
                              [{file,"supervisor.erl"},{line,301}]},
                          {gen_server,init_it,2,
                              [{file,"gen_server.erl"},{line,417}]},
                          {gen_server,init_it,6,
                              [{file,"gen_server.erl"},{line,385}]},
                          {proc_lib,init_p_do_apply,3,
                              [{file,"proc_lib.erl"},{line,226}]}]},
                     {eimp_app,start,[normal,[]]}}
      in function  application_master:init/4 (application_master.erl, line 138)
    ancestors: [<0.178.0>]
    message_queue_len: 1
    messages: [{'EXIT',<0.180.0>,normal}]
    links: [<0.178.0>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 28
    reductions: 221
  neighbours:

2022-04-28 04:41:06.998647+00:00 [notice] <0.44.0>@application_controller:info_exited/3:1943     application: eimp
    exited: {{function_clause,
                 [{lists,seq,[1,unknown],[{file,"lists.erl"},{line,243}]},
                  {eimp_sup,init,1,[{file,"eimp_sup.erl"},{line,45}]},
                  {supervisor,init,1,[{file,"supervisor.erl"},{line,301}]},
                  {gen_server,init_it,2,[{file,"gen_server.erl"},{line,417}]},
                  {gen_server,init_it,6,[{file,"gen_server.erl"},{line,385}]},
                  {proc_lib,init_p_do_apply,3,
                      [{file,"proc_lib.erl"},{line,226}]}]},
             {eimp_app,start,[normal,[]]}}
    type: temporary

2022-04-28 04:41:06.998896+00:00 [info] <0.120.0>@ejabberd_config:load/1:82 Loading configuration from /etc/ejabberd/ejabberd.yml
2022-04-28 04:41:07.005464+00:00 [notice] <0.120.0>@ejabberd_logger:set/1:367 Changing loglevel from 'info' to 'warning'
2022-04-28 04:41:07.568889+00:00 [critical] <0.339.0>@gen_mod:start_module/4:169 Failed to start module mod_muc:
{error,
    {{function_clause,
         [{lists,seq,[1,unknown],[{file,"lists.erl"},{line,243}]},
          {mod_muc_sup,init,1,[{file,"mod_muc_sup.erl"},{line,63}]},
          {supervisor,init,1,[{file,"supervisor.erl"},{line,301}]},
          {gen_server,init_it,2,[{file,"gen_server.erl"},{line,417}]},
          {gen_server,init_it,6,[{file,"gen_server.erl"},{line,385}]},
          {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]},
     {child,undefined,'mod_muc_sup_nixnet.services',
         {mod_muc_sup,start_link,[<<"nixnet.services">>]},
         permanent,infinity,supervisor,
         [mod_muc_sup]}}}
2022-04-28 04:41:07.569247+00:00 [critical] <0.339.0>@gen_mod:maybe_halt_ejabberd/0:256 ejabberd initialization was aborted because a module start failed.
2022-04-28 04:41:07.568623+00:00 [error] <0.428.0>@proc_lib:crash_report/4:525 CRASH REPORT:
  crasher:
    initial call: supervisor:mod_muc_sup/1
    pid: <0.428.0>
    registered_name: []
    exception error: no function clause matching lists:seq(1,unknown) (lists.erl, line 243)
      in function  mod_muc_sup:init/1 (mod_muc_sup.erl, line 63)
      in call from supervisor:init/1 (supervisor.erl, line 301)
      in call from gen_server:init_it/2 (gen_server.erl, line 417)
      in call from gen_server:init_it/6 (gen_server.erl, line 385)
    ancestors: [ejabberd_gen_mod_sup,ejabberd_sup,<0.120.0>]
    message_queue_len: 0
    messages: []
    links: [<0.411.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 28
    reductions: 234
  neighbours:

Last updated: May 5, 2022 at 4:47 AM UTC