Logscape Support

Members Login
Username 
 
Password 
    Remember Me  
Logscape Support -> logscape -> Forwarder seems to be running on the wrong port
Post Info TOPIC: Forwarder seems to be running on the wrong port
cy83Rsh4rk


Member

Status: Offline
Posts: 5
Date:
Forwarder seems to be running on the wrong port
Permalink  
 

Forgive me if this is something easy but I'm new to logscape. My understanding is that both the manager and the forwarder should run on port 11000. Howerver, my forwarder seems to be running on port 11003 and it can't connect to the manager so it dies.

Here's a snippet of my agent.log...

 

2015-03-07 19:27:15,159 ERROR main (proxy.ProxyClient)  Failing to make call:interface com.liquidlabs.vso.resource.ResourceSpace.getSystemResourceId Errors:8

2015-03-07 19:27:15,298 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003 Unknown Rcvr[DeploymentService_StartAgent32236@user-service-5].public abstract void com.liquidlabs.transport.proxy.AddressUpdater.syncEndPoints(java.lang.String[],java.lang.String[]) throws java.lang.Exception / client:stcp://172.16.4.171:11000

2015-03-07 19:27:15,298 INFO netty-reply-1-1 (proxy.PeerHandler)         Failed to pass on event, srcId[DeploymentService_StartAgent32236@user-service-5syncEndPoints--1907496b:14bf5964541:-68ae-4881] destId[DeploymentService_StartAgent32236@user-service-5]

2015-03-07 19:27:15,298 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003 methodInvokerIds:{LookupSpace__HA_VSOAgent32482@user-service-5, WorkAllocator_StartAgent32482@user-service-5, ResourceSpace_StartAgent32482@user-service-5, DeploymentService_StartAgent32482@user-service-5, ResourceAgent}

2015-03-07 19:27:15,298 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003 notifyInvokerIds:{}

2015-03-07 19:27:15,298 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003**** SENDING ERROR BACK TO CALLER ****** :stcp://172.16.4.171:11000.public abstract void com.liquidlabs.transport.proxy.AddressUpdater.syncEndPoints(java.lang.String[],java.lang.String[]) throws java.lang.Exception

2015-03-07 19:27:15,298 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003?svc=SHARED ex:java.lang.RuntimeException: stcp://172.16.5.15:11003 ReceiverIdNotFound:DeploymentService_StartAgent32236@user-service-5 = available{LookupSpace__HA_VSOAgent32482@user-service-5,

        WorkAllocator_StartAgent32482@user-service-5,

        ResourceSpace_StartAgent32482@user-service-5,

        DeploymentService_StartAgent32482@user-service-5,

        ResourceAgent}

2015-03-07 19:27:15,301 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003 Bad invocation suppressed msg:DeploymentService_StartAgent32236@user-service-5 BadInvocations:1

2015-03-07 19:27:15,301 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003**** SENDING ERROR BACK TO CALLER ****** :stcp://172.16.4.171:11000.public abstract void com.liquidlabs.transport.proxy.AddressUpdater.syncEndPoints(java.lang.String[],java.lang.String[]) throws java.lang.Exception

2015-03-07 19:27:15,301 WARN netty-reply-1-1 (proxy.PeerHandler)        stcp://172.16.5.15:11003?svc=SHARED ex:java.lang.RuntimeException: stcp://172.16.5.15:11003 ReceiverIdNotFound:DeploymentService_StartAgent32236@user-service-5 = available{LookupSpace__HA_VSOAgent32482@user-service-5,

        WorkAllocator_StartAgent32482@user-service-5,

        ResourceSpace_StartAgent32482@user-service-5,

        DeploymentService_StartAgent32482@user-service-5,

        ResourceAgent}

2015-03-07 19:27:16,159 ERROR main (proxy.ProxyClient)  ERROR:[KeepOrderedAddresser:ResourceSpace/interface com.liquidlabs.vso.resource.ResourceSpace /: list[] badEP[stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0] replay?:true reason:Registered Failure]

2015-03-07 19:27:16,159 INFO main (addressing.KeepOrderedAddresser)     ResourceSpace/interface com.liquidlabs.vso.resource.ResourceSpace Refreshing Endpoints existingEPS:[]

2015-03-07 19:27:16,160 WARN main (vso.SpaceServiceImpl)        _HA_>>>>>>>>>  Trying to Refresh Endpoints for:ResourceSpace from:ProxyClient/-979051605 id:LookupSpace h:-979051605 fact:ProxyFactoryImpl@54495403 stcp://172.16.5.15:11003/?svc=SHARED&host=user-service-5&_startTime=07-Mar-15_19-26-56&udp=0 if:LookupSpace,LifeCycle,Leasor [KeepOrderedAddresser:LookupSpace/interface com.liquidlabs.vso.lookup.LookupSpace /: list[stcp://172.16.4.171:11000/?svc=LookupSpace&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-05&udp=0] badEP[] replay?:false reason:] STARTED LastUsed:07-Mar-15 19:27:14,155 LastSync:07-Mar-15 19:27:05,273 LastRefresh:

2015-03-07 19:27:16,162 WARN main (vso.SpaceServiceImpl)        _HA_<<<<<<<<< Given EPS:[stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0]

2015-03-07 19:27:16,662 WARN main (netty.NettySenderFactoryProxy)       Sender[noSender] Ex[ChannelFuture.connectFailed:stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0] Stats[NettySendFactory senderPool:2 live:0 allSenders:2] URI[stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0] Info[getSystemResourceId] LiveConnections:0

2015-03-07 19:27:17,162 WARN main (proxy.ProxyClient)   RetryException:stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0 ex:com.liquidlabs.transport.proxy.RetryInvocationException: SendFailed.Throwable:noSender

com.liquidlabs.transport.proxy.RetryInvocationException: SendFailed.Throwable:noSender

        at com.liquidlabs.transport.netty.NettySenderFactoryProxy.send(NettySenderFactoryProxy.java:95)

        at com.liquidlabs.transport.netty.NettyEndPoint.send(NettyEndPoint.java:96)

        at com.liquidlabs.transport.proxy.PeerHandler.send(PeerHandler.java:380)

        at com.liquidlabs.transport.proxy.ProxyClient.sendInvocation(ProxyClient.java:548)

        at com.liquidlabs.transport.proxy.ProxyClient.clientExecute(ProxyClient.java:501)

        at com.liquidlabs.transport.proxy.ProxyClient.sendWithRetry(ProxyClient.java:460)

        at com.liquidlabs.transport.proxy.ProxyClient.invoke(ProxyClient.java:219)

        at com.sun.proxy.$Proxy8.getSystemResourceId(Unknown Source)

        at com.liquidlabs.vso.agent.ResourceAgentImpl.getSystemResourceId(ResourceAgentImpl.java:403)

        at com.liquidlabs.vso.agent.ResourceAgentImpl.start(ResourceAgentImpl.java:168)

        at com.liquidlabs.vso.VSOMain.startAgent(VSOMain.java:318)

        at com.liquidlabs.vso.VSOMain.agent(VSOMain.java:199)

        at com.liquidlabs.vso.VSOMain.main(VSOMain.java:131)

Caused by: java.lang.RuntimeException: ChannelFuture.connectFailed:stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0

        at com.liquidlabs.transport.netty.NettySenderImpl.start(NettySenderImpl.java:198)

        at com.liquidlabs.transport.netty.NettyPoolingSenderFactory.getSender(NettyPoolingSenderFactory.java:110)

        at com.liquidlabs.transport.netty.NettySenderFactoryProxy.getSender(NettySenderFactoryProxy.java:102)

        at com.liquidlabs.transport.netty.NettySenderFactoryProxy.send(NettySenderFactoryProxy.java:53)

        ... 12 more

2015-03-07 19:27:17,163 ERROR main (proxy.ProxyClient)  Failing to make call:interface com.liquidlabs.vso.resource.ResourceSpace.getSystemResourceId Errors:9

2015-03-07 19:27:18,164 INFO Thread-4 (process.ProcessHandler)  ShutHook - Destroying outstanding processes, count:0

2015-03-07 19:27:18,164 INFO Thread-1 (transport.TransportFactory)      com.liquidlabs.transport.TransportFactoryImpl@5ce81285 Shutdown hook called

2015-03-07 19:27:18,165 INFO Thread-1 (transport.TransportFactory)      Shutting down now...

2015-03-07 19:27:21,166 INFO Thread-1 (transport.TransportFactory)      EPs killed:2

 

Any help would be greatly appreciated.

 

Thanks!



__________________
Nathen


Member

Status: Offline
Posts: 23
Date:
Permalink  
 

Hi cy83Rsh4rk,

Once you have a Logscape manager running you can use the configure script to have any new agents set up for you.

On your forwarder navigate to the "logscape/scripts" directory, where you'll see the configure.sh and configure.bat, execute whichever is appropriate for your system, and you'll be prompted for answers to a few questions, once complete your agent will be configured and can be started in the normal way.

Hope this helps,
Nathen

__________________
cy83Rsh4rk


Member

Status: Offline
Posts: 5
Date:
Permalink  
 

Hi Nathen,
Thanks for the reply.

I ran the configure.sh file and didn't specify a port (Just hit enter for the default 11000 port) and I'm still getting the error above. I double checked and I can connect to the Manager server on port 11000.



[root@user-service-3 ~]# cd /usr/local/logscape/scripts/
[root@user-service-3 scripts]# ./configure.sh
** Run from logscape/scripts: i.e. >./configure.sh **

Welcome to the LogScape Installer
---------------------------------

Which type/role of agent would you like to configure?
1. dev.Management (Runs Management Services and WebConsole on default port:8080) (includes:SocketServer,IndexStore,SysLogServer)
2. dev.Forwarder (Forward to Manager OR dedicated IndexStore instance)
3. dev.IndexStore (Receive forwarded data streams) - already run on Management node)
4. dev.Indexer (Index & Search)
5. dev.Failover (Run Failover instance - manual config of boot.properties needed)
(1-5) or enter another role <zone>.<subzone>.<role> (ending with Server or Indexer)? :2

General Configuration
---------------------

What is the Hostname/IpAddress where the LogScape Management is running: 172.16.4.171

Enter the Management port [Enter to accept 11000]?

Proceeding with Network connectivity check....
Opening Server:user-service-3.localdomain:11000

IPInfo:127.0.1.1:11000

Starting Client on:user-service-3.localdomain Requesting msgs from:172.16.4.171
ReceivedMSG: stcp://172.16.4.171:11000 1 Write from client:Sat Mar 21 21:32:27 UTC 2015 to:172.16.5.86_33337

ReceivedMSG: stcp://172.16.4.171:11000 2 Write from client:Sat Mar 21 21:32:28 UTC 2015 to:172.16.5.86_33337

ReceivedMSG: stcp://172.16.4.171:11000 3 Write from client:Sat Mar 21 21:32:29 UTC 2015 to:172.16.5.86_33337

Socket Problem: [Socket closed] when connecting to host connect to Host:172.16.4.171:11000

Test:SUCCESS - we received:3 messages from:172.16.4.171:11000

** NetworkCheck was SUCCESSFUL **


--------------------- Values Entered ------
AgentType:dev.Forwarder
WebPort:8080
Management:172.16.4.171:11000
Creating back-up of setup.conf
logscape_home=/usr/local/logscape
Run from >logscape/scripts: i.e. ./configure.sh
Working Dir:./..
Using Config:etc/setup.conf
Path:boot.properties:etc/logscape-service-x86.ini:etc/logscape-service-x64.ini:conf/agent.conf:conf/lookup.conf:logscape.sh:logscape.bat
The Logscape Agent can now be started using 'logscape.sh start' Credentials: (admin/ll4bs)


Any ideas?


Thanks again!

__________________
cy83Rsh4rk


Member

Status: Offline
Posts: 5
Date:
Permalink  
 

Actually, I just realized that the Forwarder is just trying to connect to the Manager on the wrong port. I just don't know how it determines to use port 11003 even after I select 11000 when running the config.sh

__________________
Nathen


Member

Status: Offline
Posts: 23
Date:
Permalink  
 

Hi cy83Rsh4rk,

If you run your manager, and navigate to the agents page, and then configure and run your forwarder does the forwarder appear in the list of agents?

You should see something similar to this on the agent page, where it shows the agents name, ip and role.

UvDEfov.png


Also what operating system and Java version is on the machine of both the forwarder and manager?

Thanks,
Nathen



-- Edited by Nathen on Tuesday 24th of March 2015 09:40:32 AM

__________________
cy83Rsh4rk


Member

Status: Offline
Posts: 5
Date:
Permalink  
 

Hi Nathen,
Thank you for the reply. When I log into my manager I don't see the forwarder listed because when I start the forwarded it tries to connect on port 11003 (or at least that is what it seems like from the error below - maybe I'm reading it wrong).

Manager IP: 172.16.4.171
Forwarder: 172.16.5.25


Here's a snippet of the agent.log file...

2015-03-27 15:38:16,807 WARN main (vso.SpaceServiceImpl) _HA_>>>>>>>>> Trying to Refresh Endpoints for:ResourceSpace from:ProxyClient/-979051605 id:LookupSpace h:-979051605 fact:ProxyFactoryImpl@54495403 stcp://172.16.5.25:11003/?svc=SHARED&host=invoice-service-3&_startTime=27-Mar-15_15-38-07&udp=0 if:LookupSpace,LifeCycle,Leasor [KeepOrderedAddresser:LookupSpace/interface com.liquidlabs.vso.lookup.LookupSpace /: list[stcp://172.16.4.171:11000/?svc=LookupSpace&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-05&udp=0] badEP[] replay?:false reason:] STARTED LastUsed:27-Mar-15 15:38:14,802 LastSync: LastRefresh:
2015-03-27 15:38:16,809 WARN main (vso.SpaceServiceImpl) _HA_<<<<<<<<< Given EPS:[stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0]
2015-03-27 15:38:17,310 WARN main (netty.NettySenderFactoryProxy) Sender[noSender] Ex[ChannelFuture.connectFailed:stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0] Stats[NettySendFactory senderPool:2 live:0 allSenders:2] URI[stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0] Info[getSystemResourceId] LiveConnections:0
2015-03-27 15:38:17,811 WARN main (proxy.ProxyClient) RetryException:stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0 ex:com.liquidlabs.transport.proxy.RetryInvocationException: SendFailed.Throwable:noSender
com.liquidlabs.transport.proxy.RetryInvocationException: SendFailed.Throwable:noSender
at com.liquidlabs.transport.netty.NettySenderFactoryProxy.send(NettySenderFactoryProxy.java:95)
at com.liquidlabs.transport.netty.NettyEndPoint.send(NettyEndPoint.java:96)
at com.liquidlabs.transport.proxy.PeerHandler.send(PeerHandler.java:380)
at com.liquidlabs.transport.proxy.ProxyClient.sendInvocation(ProxyClient.java:548)
at com.liquidlabs.transport.proxy.ProxyClient.clientExecute(ProxyClient.java:501)
at com.liquidlabs.transport.proxy.ProxyClient.sendWithRetry(ProxyClient.java:460)
at com.liquidlabs.transport.proxy.ProxyClient.invoke(ProxyClient.java:219)
at com.sun.proxy.$Proxy8.getSystemResourceId(Unknown Source)
at com.liquidlabs.vso.agent.ResourceAgentImpl.getSystemResourceId(ResourceAgentImpl.java:403)
at com.liquidlabs.vso.agent.ResourceAgentImpl.start(ResourceAgentImpl.java:168)
at com.liquidlabs.vso.VSOMain.startAgent(VSOMain.java:318)
at com.liquidlabs.vso.VSOMain.agent(VSOMain.java:199)
at com.liquidlabs.vso.VSOMain.main(VSOMain.java:131)
Caused by: java.lang.RuntimeException: ChannelFuture.connectFailed:stcp://172.16.4.171:11003/?svc=SHARED&host=ip-172-16-4-171&_startTime=07-Mar-15_18-54-01&udp=0
at com.liquidlabs.transport.netty.NettySenderImpl.start(NettySenderImpl.java:198)
at com.liquidlabs.transport.netty.NettyPoolingSenderFactory.getSender(NettyPoolingSenderFactory.java:110)
at com.liquidlabs.transport.netty.NettySenderFactoryProxy.getSender(NettySenderFactoryProxy.java:102)
at com.liquidlabs.transport.netty.NettySenderFactoryProxy.send(NettySenderFactoryProxy.java:53)
... 12 more


I am running java 1.8.0_25. I know your site says it doesn't work on java 8 but I actually installed it on another system running java 8 and it just worked.

Thanks again for your help!



__________________
Nathen


Member

Status: Offline
Posts: 23
Date:
Permalink  
 

Hi cy83Rsh4rk,

Unfortunately Logscape 2.5 doesn't support java 1.8.X so that is likely to root cause of your issue.

Your agent will not be booting properly due to the java incompatibility.

The port error you're seeing as related to the agent processes - as can be seen here.

Installing JRE7 and running Logscape should rectify your problem.



__________________
cy83Rsh4rk


Member

Status: Offline
Posts: 5
Date:
Permalink  
 

Hi Nathen,
Thank you SOOO much for your help. The issue I was having was that I only had port 11000 opened on both the manager and the forwarder. Once I opened 11000-11100 on both it started working. BTW, I'm running on java 1.8 and it seems to work


Thanks again for all your help!!!!!!!

__________________
Page 1 of 1  sorted by
 
Quick Reply

Please log in to post quick replies.
Logscape Support -> logscape -> Forwarder seems to be running on the wrong port


Create your own FREE Forum
Report Abuse 		Powered by ActiveBoard