<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Michael,<div class=""><br class=""></div><div class="">Glad you found this.</div><div class=""><br class=""></div><div class="">The openssl library is a nightmare. A lot of distributions rigidly stick to one version for this kind of reason. Like libc, glibc, etc. Updating it can be scary.</div><div class=""><br class=""></div><div class="">Perl does have some powerful dependency checking and versioning (even including scripting capability on the imported module version number), but it is rare to see developers use it.</div><div class=""><br class=""></div><div class="">Regards, Mark.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 16 Dec 2020, at 5:42 AM, Michael Balzer <<a href="mailto:dexter@expeedo.de" class="">dexter@expeedo.de</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="content-isolator__container"><div class="protected-part"><div class="protected-title">Signed PGP part</div><div class="protected-content">
  
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
  
  <div class="">
    Found & fixed it.<br class="">
    <br class="">
    Grepping for apns in the log wasn't sufficient, the relevant hint
    was on the output line following the log entry:<br class="">
    <br class="">
    <font face="monospace" class="">2020-12-15 20:01:10.969115 +0100 info  main:
      - - - msg apns connected to <a href="http://gateway.sandbox.push.apple.com" class="">gateway.sandbox.push.apple.com</a>, now
      establishing SSL security<br class="">
      EV: error in callback (ignoring): Your vendor has not defined
      SSLeay macro ST_OK at ovms_server.pl line 1683.</font><br class="">
    <br class="">
    Looking for this on the web, I found this 2018 bug report on
    AnyEvent::Handle vs. OpenSSL:<br class="">
    <br class="">
    <a class="moz-txt-link-freetext" href="https://rt.cpan.org/Public/Bug/Display.html?id=124723">https://rt.cpan.org/Public/Bug/Display.html?id=124723</a><br class="">
    <br class="">
    The perl test command showed the exact error, so I applied the
    linked patch to the AnyEvent::Handle module, and voila, it's working
    again.<br class="">
    <br class="">
    Strange this only affected the APNS connection.<br class="">
    <br class="">
    The cause was probably the perl-OpenSSL update on my server on
    2020-10-01, which came after the latest AnyEvent update
    (2020-09-18). I have to admit, over the years I grew a bit of hatred
    for perl, from a server management point of view. No other
    subsystems on my servers (with the exception of python of course)
    have had so many issues with incompatible package updates.<br class="">
    <br class="">
    Regards,<br class="">
    Michael<br class="">
    <br class="">
    <br class="">
    <div class="moz-cite-prefix">Am 15.12.20 um 07:55 schrieb Mark
      Webb-Johnson:<br class="">
    </div>
    <blockquote type="cite" cite="mid:F1BA7BBC-7D51-46CC-A02E-FEDF609B0EB4@webb-johnson.net" class="">
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
      I think perhaps the best is to add an overall timer, launched just
      before the tcp_connect, and cancelled in all the known exit paths.
      Then that timer (perhaps 60 seconds) could cleanup the push. At
      least that would avoid the whole system jamming up.
      <div class=""><br class="">
      </div>
      <div class="">But that doesn’t solve the core problem of why you
        can’t connect to apple (but I can). Your tcp connection state is
        “UNCONN”, which means the disconnected, I assume.</div>
      <div class=""><br class="">
      </div>
      <div class="">I think there is an AnyEvent debug setting, but that
        is likely to produce a lot of verbose output on a production
        server.</div>
      <div class=""><br class="">
      </div>
      <div class="">Regards, Mark,<br class="">
        <div class=""><br class="">
          <blockquote type="cite" class="">
            <div class="">On 15 Dec 2020, at 4:20 AM, Michael Balzer
              <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>> wrote:</div>
            <br class="Apple-interchange-newline">
            <div class="">
              <div class="content-isolator__container">
                <div class="protected-part">
                  <div class="protected-title">Signed PGP part</div>
                  <div class="protected-content">
                    <meta http-equiv="Content-Type" content="text/html;
                      charset=UTF-8" class="">
                    <div class=""> The certificates are those you sent
                      me. Just checked, they're both valid.<br class="">
                      <br class="">
                      I've added your suggestion and also added the
                      "connected" log message from PushAPNS.pm, no luck.<br class="">
                      <br class="">
                      <font class="" face="monospace">2020-12-14
                        13:52:00.840238 +0100 info  main: - - XXXXXXXX
                        msg queued apns notification for
                        sandbox:XXXXXXXXXXXXXX<br class="">
                        2020-12-14 13:52:01.579649 +0100 info  main: - -
                        - msg apns processing queue for <a href="http://gateway.sandbox.push.apple.com/" class="" moz-do-not-send="true">gateway.sandbox.push.apple.com</a><br class="">
                        2020-12-14 13:52:01.740147 +0100 info  main: - -
                        - msg apns connected to <a href="http://gateway.sandbox.push.apple.com/" class="" moz-do-not-send="true">gateway.sandbox.push.apple.com</a>,
                        now establishing SSL security<br class="">
                      </font><br class="">
                      I tried reducing the APNS channels to
                      "production", still no luck.<br class="">
                      <br class="">
                      I can see the socket getting created when
                      following socket events:<br class="">
                      <br class="">
                      <font class="" face="monospace">[root@ns34 ~]# ss
                        -E | grep "17\.188"<br class="">
                        tcp    UNCONN  0        0       
                        146.0.237.226:59102        17.188.136.189:2195<br class="">
                      </font><br class="">
                      So the TLS init somehow fails in a way
                      AnyEvent::Handle doesn't recognize as a failure /
                      error.<br class="">
                      <br class="">
                      I've had a look at the AnyEvent::Handle
                      documentation but cannot see anything we're doing
                      wrong or missed regarding error handling.<br class="">
                      <br class="">
                      Maybe Apple is blocking me, dropping all packets?
                      But wouldn't that trigger a timeout or TLS init
                      error?<br class="">
                      <br class="">
                      Very strange. Any other ideas?<br class="">
                      <br class="">
                      Regards,<br class="">
                      Michael<br class="">
                      <br class="">
                      <br class="">
                      <div class="moz-cite-prefix">Am 14.12.20 um 09:19
                        schrieb Mark Webb-Johnson:<br class="">
                      </div>
                      <blockquote type="cite" cite="mid:9EAE2052-E79A-4A36-87C3-29FAE4A3D847@webb-johnson.net" class="">
                        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
                        Do you have conf/ovms_apns_sandbox.pem file in
                        place? Valid and not expired?
                        <div class=""><br class="">
                        </div>
                        <div class="">I had a quick review, and it seems
                          the main flow handles errors. Perhaps some
                          other callback on the AnyEvent::Handle for an
                          error condition is being missed? Or perhaps
                          the AnyEvent::Handle could not be created at
                          all. Can you try to add:</div>
                        <div class=""><br class="">
                        </div>
                        <blockquote style="margin: 0 0 0 40px; border:
                          none; padding: 0px;" class="">
                          <div class="">If (!defined $apns_handle)</div>
                          <div class="">  {</div>
                          <div class="">  AE::log error => "- - - msg
                            apns handle could not be created”;</div>
                          <div class="">  $apns_running = 0;</div>
                          <div class="">  }</div>
                        </blockquote>
                        <div class="">
                          <div class=""><br class="">
                          </div>
                          <div class="">After the block of ‘$apns_handle
                            = new AnyEvent::Handle(…’?</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">Regards, Mark.</div>
                          <div class=""><br class="">
                            <blockquote type="cite" class="">
                              <div class="">On 14 Dec 2020, at 3:21 PM,
                                Michael Balzer <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>>
                                wrote:</div>
                              <br class="Apple-interchange-newline">
                              <div class="">
                                <div class="content-isolator__container">
                                  <div class="protected-part">
                                    <div class="protected-title">Signed
                                      PGP part</div>
                                    <div class="protected-content">
                                      <meta http-equiv="Content-Type" content="text/html;
                                        charset=UTF-8" class="">
                                      <div class=""> That's the strange
                                        part: I don't get any error, and
                                        I don't get a timeout either.
                                        That's what I meant by "fails in
                                        a way we don't handle".<br class="">
                                        <br class="">
                                        This is the only log entry on
                                        "apns processing", I get this
                                        once after restarting the server
                                        as soon as the first APN is due
                                        for delivery:<br class="">
                                        <font class="" face="monospace"><br class="">
                                          2020-12-10 16:55:30.351530
                                          +0100 info  main: - - - msg
                                          apns processing queue for <a href="http://gateway.sandbox.push.apple.com/" class="" moz-do-not-send="true">gateway.sandbox.push.apple.com</a></font><br class="">
                                        <br class="">
                                        (still running server v2 due to
                                        lack of time)<br class="">
                                        <br class="">
                                        After that, no more apns
                                        processing – I guess because
                                        $apns_running never gets reset.<br class="">
                                        <br class="">
                                        Any idea?<br class="">
                                        <br class="">
                                        Regards,<br class="">
                                        Michael<br class="">
                                        <br class="">
                                        <br class="">
                                        <div class="moz-cite-prefix">Am
                                          14.12.20 um 07:17 schrieb Mark
                                          Webb-Johnson:<br class="">
                                        </div>
                                        <blockquote type="cite" cite="mid:FFACA9C0-DE7C-45A2-BD55-E6B8B0E40462@webb-johnson.net" class="">
                                          <meta http-equiv="Content-Type" content="text/html;
                                            charset=UTF-8" class="">
                                          <div class=""><br class="">
                                          </div>
                                          Push notifications are still
                                          working ok for me. What is the
                                          error you get back from the
                                          gateway?
                                          <div class=""><br class="">
                                          </div>
                                          <div class="">We are using
                                            this protocol (see apns_send
                                            function):</div>
                                          <div class=""><br class="">
                                          </div>
                                          <blockquote style="margin: 0 0
                                            0 40px; border: none;
                                            padding: 0px;" class="">
                                            <div class=""><a href="https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/BinaryProviderAPI.html" class="" moz-do-not-send="true">https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/BinaryProviderAPI.html</a></div>
                                          </blockquote>
                                          <div class="">
                                            <div class=""><br class="">
                                            </div>
                                            <div class="">We need to
                                              convert to this one:</div>
                                            <div class=""><br class="">
                                            </div>
                                          </div>
                                          <blockquote style="margin: 0 0
                                            0 40px; border: none;
                                            padding: 0px;" class="">
                                            <div class="">
                                              <div class=""><a href="https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/sending_notification_requests_to_apns/" class="" moz-do-not-send="true">https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/sending_notification_requests_to_apns/</a></div>
                                            </div>
                                          </blockquote>
                                          <div class="">
                                            <div class="">
                                              <div class=""><br class="">
                                              </div>
                                              <div class="">But have
                                                until the end of March
                                                2021 to do it. The
                                                switch is not complex,
                                                and is probably slightly
                                                easier for us. It
                                                becomes just a simple
                                                http request, with
                                                pretty much the same
                                                payload we currently use
                                                (in json format).</div>
                                              <div class=""><br class="">
                                              </div>
                                              <div class="">Regards,
                                                Mark.</div>
                                              <div class=""><br class="">
                                                <blockquote type="cite" class="">
                                                  <div class="">On 11
                                                    Dec 2020, at 12:27
                                                    AM, Michael Balzer
                                                    <<a href="mailto:dexter@expeedo.de" class="" moz-do-not-send="true">dexter@expeedo.de</a>>
                                                    wrote:</div>
                                                  <br class="Apple-interchange-newline">
                                                  <div class="">
                                                    <div class="">
                                                      <div class="content-isolator__container">
                                                        <div class="protected-part">
                                                          <div class="protected-title">Signed
                                                          PGP part</div>
                                                          <div class="protected-content">Mark,<br class="">
                                                          <br class="">
                                                          a user
                                                          informed me he
                                                          no longer gets
                                                          any push
                                                          notifications
                                                          to iOS.<br class="">
                                                          <br class="">
                                                          Looking into
                                                          the logs, it
                                                          seems the
                                                          initial
                                                          connect to the
                                                          gateway fails
                                                          in a way the
                                                          perl code does
                                                          not handle.<br class="">
                                                          <br class="">
                                                          I've found
                                                          this in the
                                                          Apple forums:
                                                          <a href="https://developer.apple.com/forums/thread/667248" class="" moz-do-not-send="true">https://developer.apple.com/forums/thread/667248</a><br class="">
                                                          <br class="">
                                                          It seems the
                                                          protocol we
                                                          use has been
                                                          deprecated,
                                                          but it should
                                                          continue to
                                                          work until
                                                          March. Do you
                                                          see a similar
                                                          effect on your
                                                          server?<br class="">
                                                          <br class="">
                                                          Regards,<br class="">
                                                          Michael<br class="">
                                                          <br class="">
                                                          -- <br class="">
                                                          Michael Balzer
                                                          * Helkenberger
                                                          Weg 9 *
                                                          D-58256
                                                          Ennepetal<br class="">
                                                          Fon 02333 /
                                                          833 5735 *
                                                          Handy 0176 /
                                                          206 989 26<br class="">
                                                          <br class="">
                                                          <br class="">
                                                          </div>
                                                        </div>
                                                        <br class="">
                                                        <iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg==" width="200" height="10"></iframe></div>
                                                    </div>
                                                  </div>
                                                </blockquote>
                                              </div>
                                              <br class="">
                                            </div>
                                          </div>
                                          <br class="">
                                          <fieldset class="mimeAttachmentHeader"></fieldset>
                                          <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                                        </blockquote>
                                        <br class="">
                                        <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                                      </div>
                                    </div>
                                  </div>
                                  <br class="">
                                  <iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg==" width="200" height="10"></iframe></div>
                              </div>
                            </blockquote>
                          </div>
                          <br class="">
                        </div>
                        <br class="">
                        <fieldset class="mimeAttachmentHeader"></fieldset>
                        <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
                      </blockquote>
                      <br class="">
                      <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
                    </div>
                  </div>
                </div>
                <br class="">
                <iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg==" width="200" height="10"></iframe></div>
            </div>
          </blockquote>
        </div>
        <br class="">
      </div>
      <br class="">
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
    </blockquote>
    <br class="">
    <pre class="moz-signature" cols="72">-- 
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
  </div>

</div></div><br class=""><iframe class="content-isolator__isolated-content" sandbox="allow-scripts" scrolling="auto" width="200" height="10" style="border:none;display:block;overflow:auto;" data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="></iframe></div></div></blockquote></div><br class=""></div></body></html>