vCenter Appliance 6.7 Update 3 (6.7.0.40000) cause une alarme : VMware vAPI Endpoint

Bonjour à toutes et à tous,

Lors de l’upgrade de mon appliance vers la version 6.7 Update 3 (6.7.0.40000) j’ai été confronté à un message d’erreur me disant que le composant VMware vAPI Endpoint rencontrait une erreur. Une alarme orange donc.

En me connectant en ssh sur l’appliance j’ai pu voir que les logs du service en question (situés ici : /var/log/vmware/vapi/endpoint/endpoint.log) comportait le message suivant :

2020-04-02T18:06:31.433Z | WARN | state-manager1 | ApiInterfacesFactory | Retrieving interfaces for service cac1c396-bfbf-41f8-b704-5f049c3bbed8\com.vmware.applmgmt has failed.
com.vmware.vapi.client.exception.ConnectionException: http://localhost:8201/api invocation failed with "java.net.ConnectException: Connection refused"
at com.vmware.vapi.internal.protocol.client.rpc.http.ApacheHttpAsyncClientTransport$FutureCallbackImpl.failed(ApacheHttpAsyncClientTransport.java:285)
at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:130)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.connectionRequestFailed(DefaultClientExchangeHandlerImpl.java:319)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.access$100(DefaultClientExchangeHandlerImpl.java:63)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl$1.failed(DefaultClientExchangeHandlerImpl.java:365)
at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:130)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalPoolEntryCallback.failed(PoolingNHttpClientConnectionManager.java:474)
at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:130)
at org.apache.http.nio.pool.RouteSpecificPool.failed(RouteSpecificPool.java:165)
at org.apache.http.nio.pool.AbstractNIOConnPool.requestFailed(AbstractNIOConnPool.java:527)
at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.failed(AbstractNIOConnPool.java:775)
at org.apache.http.impl.nio.reactor.SessionRequestImpl.failed(SessionRequestImpl.java:154)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:175)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:147)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:189)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.doExecute(CloseableHttpAsyncClientBase.java:67)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.access$000(CloseableHttpAsyncClientBase.java:38)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:57)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:173)
… 7 more
2020-04-02T18:06:31.434Z | WARN | state-manager1 | ApiInterfacesFactory | The cache does not contain entry for service cac1c396-bfbf-41f8-b704-5f049c3bbed8\com.vmware.applmgmt. The interfaces will be skipped.
2020-04-02T18:06:31.434Z | INFO | state-manager1 | HealthStatusCollectorImpl | HEALTH YELLOW Failed to connect to cac1c396-bfbf-41f8-b704-5f049c3bbed8\com.vmware.applmgmt vAPI provider.

applmgmt

J’avais donc la raison de mon état orange sur le vcenter. Il m’était possible de me connecter via l’interface vcenter en 443 mais impossible de passer par l’endpoint https://fqdn:5480.

Après un état des services :

root@vcenter [ /etc/vmware-content-library/config ]# service-control --status
Stopped:
applmgmt vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-rbd-watchdog vmware-statsmonitor vmware-vcha vsan-dps

J’ai effectivement ce service qui est en stoppé (ce qui n’est pas normal.

Résolution :

root@vcenter [ ~ ]# service-control --start applmgmt
Operation not cancellable. Please wait for it to finish…
Performing start operation on service applmgmt…
Successfully started service applmgmt

Une fois cette action terminée, on peut voir dans les logs du VmwareApi tout redevient normal et l’alarme a disparu de l’interface web !

2020-04-02T18:31:12.719Z | INFO | state-manager1 | HealthStatusCollectorImpl | HEALTH GREEN Current vApi Endpoint health status is created between 2020-04-02T18:30:38UTC and 2020-04-02T18:31:12UTC.