Skip to content
This repository has been archived by the owner on Jun 17, 2021. It is now read-only.

High cpu and out of memory in production #72

Open
zouyx opened this issue Mar 16, 2019 · 19 comments
Open

High cpu and out of memory in production #72

zouyx opened this issue Mar 16, 2019 · 19 comments

Comments

@zouyx
Copy link
Contributor

zouyx commented Mar 16, 2019

there was high cpu usage , and high latency.
but it was turn to normal when i shut down apm.

infomation:
node client version:v1.0
thinkjs version:2.2

cpu idle:
image

@wu-sheng
Copy link
Member

What do you mean shutdown apm? Backend or agent?

And when do you shutdown? 7-10?

@zouyx
Copy link
Contributor Author

zouyx commented Mar 16, 2019

shutdown agent.

first release at 3.14 21:00.
and restart twice.
first restart at 3.15 10:00, but i haven't realized the root cause.
second time at 3.16 14:00, at the same time shutdown agent.

@wu-sheng
Copy link
Member

But, after 3.16 1400, the cpu is still high. You said the agent shutdown.

@zouyx
Copy link
Contributor Author

zouyx commented Mar 16, 2019

no , this is idle time

@wu-sheng
Copy link
Member

I see. How many throughputs of your application?

@zouyx
Copy link
Contributor Author

zouyx commented Mar 18, 2019

this app is avg 1 qps

@zouyx
Copy link
Contributor Author

zouyx commented Mar 21, 2019

image

Flame graph in v1.0

@zouyx
Copy link
Contributor Author

zouyx commented Mar 21, 2019

image

Flame graph in v1.1

@ascrutae
Copy link
Member

It seems like the CPU usage rate of V1.1 is better than V1.0's. I saw that the serialized object of GRPC Node.js costs lots of CPU time in V1.0, and I will investigate this problem.

@wu-sheng
Copy link
Member

If the QPS is just 1, why does gRPC run in such high frequency? What is your strategy of activating gRPC? @ascrutae

@ZhouBox
Copy link

ZhouBox commented Apr 10, 2019

I had the same problem, so I temporarily removed the AMP part.

@zouyx
Copy link
Contributor Author

zouyx commented Apr 10, 2019

@ZhouBox which version did you used?

@ZhouBox
Copy link

ZhouBox commented Apr 10, 2019

@zouyx 0.1.5

@zouyx
Copy link
Contributor Author

zouyx commented Apr 10, 2019

SkyAPM-nodejs 0.1.5?

i fixed this problem in 1.1.0, you can try it out

@ZhouBox
Copy link

ZhouBox commented Apr 11, 2019

@zouyx sorry, I use skyapm-egg-require 0.1.5, SkyAPM-nodejs use 1.1.2.

@zouyx
Copy link
Contributor Author

zouyx commented Jun 11, 2019

Ref #71

@wuliupo
Copy link

wuliupo commented Apr 21, 2020

+1

@zouyx
Copy link
Contributor Author

zouyx commented Apr 21, 2020

@wuliupo i think this is new version for js as below

https://github.com/apache/skywalking-client-js

@wu-sheng
Copy link
Member

@zouyx No, it isn't. That is for browser specific.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants