<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Found & fixed it.<br>
<br>
Grepping for apns in the log wasn't sufficient, the relevant hint
was on the output line following the log entry:<br>
<br>
<font face="monospace">2020-12-15 20:01:10.969115 +0100 info main:
- - - msg apns connected to gateway.sandbox.push.apple.com, now
establishing SSL security<br>
EV: error in callback (ignoring): Your vendor has not defined
SSLeay macro ST_OK at ovms_server.pl line 1683.</font><br>
<br>
Looking for this on the web, I found this 2018 bug report on
AnyEvent::Handle vs. OpenSSL:<br>
<br>
<a class="moz-txt-link-freetext" href="https://rt.cpan.org/Public/Bug/Display.html?id=124723">https://rt.cpan.org/Public/Bug/Display.html?id=124723</a><br>
<br>
The perl test command showed the exact error, so I applied the
linked patch to the AnyEvent::Handle module, and voila, it's working
again.<br>
<br>
Strange this only affected the APNS connection.<br>
<br>
The cause was probably the perl-OpenSSL update on my server on
2020-10-01, which came after the latest AnyEvent update
(2020-09-18). I have to admit, over the years I grew a bit of hatred
for perl, from a server management point of view. No other
subsystems on my servers (with the exception of python of course)
have had so many issues with incompatible package updates.<br>
<br>
Regards,<br>
Michael<br>
<br>
<br>
<div class="moz-cite-prefix">Am 15.12.20 um 07:55 schrieb Mark
Webb-Johnson:<br>
</div>
<blockquote type="cite"
cite="mid:F1BA7BBC-7D51-46CC-A02E-FEDF609B0EB4@webb-johnson.net">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
I think perhaps the best is to add an overall timer, launched just
before the tcp_connect, and cancelled in all the known exit paths.
Then that timer (perhaps 60 seconds) could cleanup the push. At
least that would avoid the whole system jamming up.
<div class=""><br class="">
</div>
<div class="">But that doesn’t solve the core problem of why you
can’t connect to apple (but I can). Your tcp connection state is
“UNCONN”, which means the disconnected, I assume.</div>
<div class=""><br class="">
</div>
<div class="">I think there is an AnyEvent debug setting, but that
is likely to produce a lot of verbose output on a production
server.</div>
<div class=""><br class="">
</div>
<div class="">Regards, Mark,<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 15 Dec 2020, at 4:20 AM, Michael Balzer
<<a href="mailto:dexter@expeedo.de" class=""
moz-do-not-send="true">dexter@expeedo.de</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="content-isolator__container">
<div class="protected-part">
<div class="protected-title">Signed PGP part</div>
<div class="protected-content">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8" class="">
<div class=""> The certificates are those you sent
me. Just checked, they're both valid.<br class="">
<br class="">
I've added your suggestion and also added the
"connected" log message from PushAPNS.pm, no luck.<br
class="">
<br class="">
<font class="" face="monospace">2020-12-14
13:52:00.840238 +0100 info main: - - XXXXXXXX
msg queued apns notification for
sandbox:XXXXXXXXXXXXXX<br class="">
2020-12-14 13:52:01.579649 +0100 info main: - -
- msg apns processing queue for <a
href="http://gateway.sandbox.push.apple.com"
class="" moz-do-not-send="true">gateway.sandbox.push.apple.com</a><br
class="">
2020-12-14 13:52:01.740147 +0100 info main: - -
- msg apns connected to <a
href="http://gateway.sandbox.push.apple.com"
class="" moz-do-not-send="true">gateway.sandbox.push.apple.com</a>,
now establishing SSL security<br class="">
</font><br class="">
I tried reducing the APNS channels to
"production", still no luck.<br class="">
<br class="">
I can see the socket getting created when
following socket events:<br class="">
<br class="">
<font class="" face="monospace">[root@ns34 ~]# ss
-E | grep "17\.188"<br class="">
tcp UNCONN 0 0
146.0.237.226:59102 17.188.136.189:2195<br
class="">
</font><br class="">
So the TLS init somehow fails in a way
AnyEvent::Handle doesn't recognize as a failure /
error.<br class="">
<br class="">
I've had a look at the AnyEvent::Handle
documentation but cannot see anything we're doing
wrong or missed regarding error handling.<br
class="">
<br class="">
Maybe Apple is blocking me, dropping all packets?
But wouldn't that trigger a timeout or TLS init
error?<br class="">
<br class="">
Very strange. Any other ideas?<br class="">
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<br class="">
<div class="moz-cite-prefix">Am 14.12.20 um 09:19
schrieb Mark Webb-Johnson:<br class="">
</div>
<blockquote type="cite"
cite="mid:9EAE2052-E79A-4A36-87C3-29FAE4A3D847@webb-johnson.net"
class="">
<meta http-equiv="Content-Type"
content="text/html; charset=UTF-8" class="">
Do you have conf/ovms_apns_sandbox.pem file in
place? Valid and not expired?
<div class=""><br class="">
</div>
<div class="">I had a quick review, and it seems
the main flow handles errors. Perhaps some
other callback on the AnyEvent::Handle for an
error condition is being missed? Or perhaps
the AnyEvent::Handle could not be created at
all. Can you try to add:</div>
<div class=""><br class="">
</div>
<blockquote style="margin: 0 0 0 40px; border:
none; padding: 0px;" class="">
<div class="">If (!defined $apns_handle)</div>
<div class=""> {</div>
<div class=""> AE::log error => "- - - msg
apns handle could not be created”;</div>
<div class=""> $apns_running = 0;</div>
<div class=""> }</div>
</blockquote>
<div class="">
<div class=""><br class="">
</div>
<div class="">After the block of ‘$apns_handle
= new AnyEvent::Handle(…’?</div>
<div class=""><br class="">
</div>
<div class="">Regards, Mark.</div>
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On 14 Dec 2020, at 3:21 PM,
Michael Balzer <<a
href="mailto:dexter@expeedo.de"
class="" moz-do-not-send="true">dexter@expeedo.de</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="content-isolator__container">
<div class="protected-part">
<div class="protected-title">Signed
PGP part</div>
<div class="protected-content">
<meta http-equiv="Content-Type"
content="text/html;
charset=UTF-8" class="">
<div class=""> That's the strange
part: I don't get any error, and
I don't get a timeout either.
That's what I meant by "fails in
a way we don't handle".<br
class="">
<br class="">
This is the only log entry on
"apns processing", I get this
once after restarting the server
as soon as the first APN is due
for delivery:<br class="">
<font class="" face="monospace"><br
class="">
2020-12-10 16:55:30.351530
+0100 info main: - - - msg
apns processing queue for <a
href="http://gateway.sandbox.push.apple.com/" class=""
moz-do-not-send="true">gateway.sandbox.push.apple.com</a></font><br
class="">
<br class="">
(still running server v2 due to
lack of time)<br class="">
<br class="">
After that, no more apns
processing – I guess because
$apns_running never gets reset.<br
class="">
<br class="">
Any idea?<br class="">
<br class="">
Regards,<br class="">
Michael<br class="">
<br class="">
<br class="">
<div class="moz-cite-prefix">Am
14.12.20 um 07:17 schrieb Mark
Webb-Johnson:<br class="">
</div>
<blockquote type="cite"
cite="mid:FFACA9C0-DE7C-45A2-BD55-E6B8B0E40462@webb-johnson.net"
class="">
<meta
http-equiv="Content-Type"
content="text/html;
charset=UTF-8" class="">
<div class=""><br class="">
</div>
Push notifications are still
working ok for me. What is the
error you get back from the
gateway?
<div class=""><br class="">
</div>
<div class="">We are using
this protocol (see apns_send
function):</div>
<div class=""><br class="">
</div>
<blockquote style="margin: 0 0
0 40px; border: none;
padding: 0px;" class="">
<div class=""><a
href="https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/BinaryProviderAPI.html"
class=""
moz-do-not-send="true">https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/BinaryProviderAPI.html</a></div>
</blockquote>
<div class="">
<div class=""><br class="">
</div>
<div class="">We need to
convert to this one:</div>
<div class=""><br class="">
</div>
</div>
<blockquote style="margin: 0 0
0 40px; border: none;
padding: 0px;" class="">
<div class="">
<div class=""><a
href="https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/sending_notification_requests_to_apns/"
class=""
moz-do-not-send="true">https://developer.apple.com/documentation/usernotifications/setting_up_a_remote_notification_server/sending_notification_requests_to_apns/</a></div>
</div>
</blockquote>
<div class="">
<div class="">
<div class=""><br class="">
</div>
<div class="">But have
until the end of March
2021 to do it. The
switch is not complex,
and is probably slightly
easier for us. It
becomes just a simple
http request, with
pretty much the same
payload we currently use
(in json format).</div>
<div class=""><br class="">
</div>
<div class="">Regards,
Mark.</div>
<div class=""><br class="">
<blockquote type="cite"
class="">
<div class="">On 11
Dec 2020, at 12:27
AM, Michael Balzer
<<a
href="mailto:dexter@expeedo.de"
class=""
moz-do-not-send="true">dexter@expeedo.de</a>>
wrote:</div>
<br
class="Apple-interchange-newline">
<div class="">
<div class="">
<div
class="content-isolator__container">
<div
class="protected-part">
<div
class="protected-title">Signed
PGP part</div>
<div
class="protected-content">Mark,<br
class="">
<br class="">
a user
informed me he
no longer gets
any push
notifications
to iOS.<br
class="">
<br class="">
Looking into
the logs, it
seems the
initial
connect to the
gateway fails
in a way the
perl code does
not handle.<br
class="">
<br class="">
I've found
this in the
Apple forums:
<a
href="https://developer.apple.com/forums/thread/667248"
class=""
moz-do-not-send="true">https://developer.apple.com/forums/thread/667248</a><br
class="">
<br class="">
It seems the
protocol we
use has been
deprecated,
but it should
continue to
work until
March. Do you
see a similar
effect on your
server?<br
class="">
<br class="">
Regards,<br
class="">
Michael<br
class="">
<br class="">
-- <br
class="">
Michael Balzer
* Helkenberger
Weg 9 *
D-58256
Ennepetal<br
class="">
Fon 02333 /
833 5735 *
Handy 0176 /
206 989 26<br
class="">
<br class="">
<br class="">
</div>
</div>
<br class="">
<iframe
class="content-isolator__isolated-content"
sandbox="allow-scripts" scrolling="auto"
style="border:none;display:block;overflow:auto;"
data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="
width="200"
height="10"></iframe></div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
<br class="">
<fieldset
class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br class="">
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</div>
</div>
</div>
<br class="">
<iframe
class="content-isolator__isolated-content"
sandbox="allow-scripts"
scrolling="auto"
style="border:none;display:block;overflow:auto;"
data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="
width="200" height="10"></iframe></div>
</div>
</blockquote>
</div>
<br class="">
</div>
<br class="">
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com" moz-do-not-send="true">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev" moz-do-not-send="true">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br class="">
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</div>
</div>
</div>
<br class="">
<iframe class="content-isolator__isolated-content"
sandbox="allow-scripts" scrolling="auto"
style="border:none;display:block;overflow:auto;"
data-src="data:text/html;charset=UTF-8;base64,PGlmcmFtZS1jb250ZW50IGRhdGEtaWZyYW1lLWhlaWdodD0idHJ1ZSI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188QlI+T3Ztc0RldiBtYWlsaW5nIGxpc3Q8QlI+T3Ztc0RldkBsaXN0cy5vcGVudmVoaWNsZXMuY29tPEJSPmh0dHA6Ly9saXN0cy5vcGVudmVoaWNsZXMuY29tL21haWxtYW4vbGlzdGluZm8vb3Ztc2RldjxCUj48L2lmcmFtZS1jb250ZW50Pg=="
width="200" height="10"></iframe></div>
</div>
</blockquote>
</div>
<br class="">
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
OvmsDev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OvmsDev@lists.openvehicles.com">OvmsDev@lists.openvehicles.com</a>
<a class="moz-txt-link-freetext" href="http://lists.openvehicles.com/mailman/listinfo/ovmsdev">http://lists.openvehicles.com/mailman/listinfo/ovmsdev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Michael Balzer * Helkenberger Weg 9 * D-58256 Ennepetal
Fon 02333 / 833 5735 * Handy 0176 / 206 989 26</pre>
</body>
</html>