complaints

Anyone else encountering extreme lag, it basicly does not move anymore for a couple of minutes when hitting train again, also entire screen goes black and des not react to a refresh

works well

Rith & Myself have been having that happen too us alot latley and ender has been notified his ' looking into it '

me too but i never thought any thing of this though

For what it's worth, things are generally fast for me. I don't really have anything in place for monitoring this sort of stuff. I do record response time in the server logs, so if a lot of people start complaining about this, I could possibly parse through the logs and see if there's been any upward trend over time or maybe specific points in time where there has been a spike.

thanks, havent had it anymore so far, it only happened at work though

I just signed up for pingdom monitoring, which can monitor one thing for free. Right now I have it monitoring the HoF page every minute.

Public stats page - http://stats.pingdom.com/y9p4la8mjlm7

nice work ender but i find the lag when training or fight more than anything

http://bots4.net/forum/3/4854 this could be the problem with me actually

Setting up monitoring for the fight/train pages is more difficult because those require authentication. Setting it up for the HoF page took all of 5 minutes to configure and should provide a fairly good indication of general website latency. So far things look fairly even.

http://bots4.net/forum/3/4854 wasn't a latency issue. With that, fighting bots with a certain battle cry/victory/defeat message would result in a blank page. I had checked the database and I'm pretty sure there was only a single bot capable of triggering this error.

It whappened to me sometimes. I was dumping with two bots in IE and the game got very slow: too much time to reload the fights, sometimes it gave errors and the other links too. I dont remember very well but I think after suspending and turning on again the pc, it was fine again. so I always thought it was from my pc, like internet conection issues...

Same to me. It happening for me from time to time. Even it happened like half hour ago. Firstly i thought is something with my connection, but any other pages opened quickly so i pinged bots4 server and it looked like ping is between 150 and 200ms so its clearly not connection issue.

/Steel Rat

The public pingdom page doesn't have the same level of granularity as the private one:

http://i.imgur.com/ZJW0YWf.png

So it seems there are occasional 30-50% average latency spikes in response time, at least for the HoF page.

Some followup questions:

1. Are you guys saying this only happens to the fight page and that when it does, other pages load fine?

2. How much lag are we talking about: a second, multiple seconds, more?

3. What kind of errors do you see, server connection errors or game errors?

4. Can you post the exact times of the page loads that are slow? The time on the top left of the page would be perfect.

Also, I parsed through the logs for Sangu's fights today. Below is the server response time (this does not account for network latency, it's just how long the server took to respond) followed by the time the request was made. It is sorted by server response time. You can see that many requests complete in the 50ms range, but that several approach the 1s mark, with one exceeding it. Does this look consistent with what you're experiencing?

6136 27/Mar/2013:07:23:37
6861 27/Mar/2013:07:03:32
7080 27/Mar/2013:07:48:47
22228 27/Mar/2013:07:14:55
33618 27/Mar/2013:07:16:16
35581 27/Mar/2013:07:16:08
35724 27/Mar/2013:07:53:14
36092 27/Mar/2013:07:49:31
36151 27/Mar/2013:07:03:18
37243 27/Mar/2013:07:30:26
37392 27/Mar/2013:07:25:25
37812 27/Mar/2013:07:13:56
38649 27/Mar/2013:07:17:42
38736 27/Mar/2013:07:45:45
39636 27/Mar/2013:07:47:28
40488 27/Mar/2013:07:36:18
40492 27/Mar/2013:08:07:38
40624 27/Mar/2013:07:52:22
40831 27/Mar/2013:08:10:40
40838 27/Mar/2013:07:56:44
41002 27/Mar/2013:07:55:31
41020 27/Mar/2013:07:31:49
41021 27/Mar/2013:08:09:49
41176 27/Mar/2013:07:50:16
41244 27/Mar/2013:08:05:17
41252 27/Mar/2013:08:01:30
41253 27/Mar/2013:07:53:31
41422 27/Mar/2013:07:31:29
41501 27/Mar/2013:08:00:22
41514 27/Mar/2013:07:44:32
41940 27/Mar/2013:07:39:26
42096 27/Mar/2013:07:49:02
42172 27/Mar/2013:08:06:02
42196 27/Mar/2013:07:29:56
42212 27/Mar/2013:07:28:23
42370 27/Mar/2013:07:44:12
42392 27/Mar/2013:07:33:27
42424 27/Mar/2013:08:01:40
42425 27/Mar/2013:07:22:30
42467 27/Mar/2013:07:41:55
42510 27/Mar/2013:07:48:55
42516 27/Mar/2013:07:44:52
42905 27/Mar/2013:07:47:07
43240 27/Mar/2013:07:51:42
43311 27/Mar/2013:07:42:17
43367 27/Mar/2013:07:49:15
43389 27/Mar/2013:07:29:11
43633 27/Mar/2013:07:19:56
43787 27/Mar/2013:07:50:58
43877 27/Mar/2013:08:11:02
44142 27/Mar/2013:07:45:59
44582 27/Mar/2013:07:38:25
44683 27/Mar/2013:07:40:49
44807 27/Mar/2013:08:04:07
44807 27/Mar/2013:08:08:18
44811 27/Mar/2013:07:28:45
44885 27/Mar/2013:08:05:28
44908 27/Mar/2013:07:52:28
44993 27/Mar/2013:08:07:21
45009 27/Mar/2013:07:23:30
45027 27/Mar/2013:08:08:58
45062 27/Mar/2013:07:30:34
45069 27/Mar/2013:07:29:26
45328 27/Mar/2013:07:08:39
45369 27/Mar/2013:07:28:53
45615 27/Mar/2013:07:16:11
45908 27/Mar/2013:07:06:16
45988 27/Mar/2013:07:04:21
45990 27/Mar/2013:07:58:24
46090 27/Mar/2013:08:11:13
46238 27/Mar/2013:07:30:40
46459 27/Mar/2013:07:50:41
46728 27/Mar/2013:07:17:23
46736 27/Mar/2013:07:58:41
46757 27/Mar/2013:07:20:45
46880 27/Mar/2013:07:19:41
46885 27/Mar/2013:08:10:01
47016 27/Mar/2013:08:03:11
47164 27/Mar/2013:07:18:41
47203 27/Mar/2013:07:14:14
47260 27/Mar/2013:07:25:35
47284 27/Mar/2013:07:26:42
47346 27/Mar/2013:07:09:37
47713 27/Mar/2013:08:11:56
47992 27/Mar/2013:08:11:30
48112 27/Mar/2013:07:48:32
48224 27/Mar/2013:07:22:02
48292 27/Mar/2013:07:47:56
48349 27/Mar/2013:07:50:29
48421 27/Mar/2013:08:08:07
48602 27/Mar/2013:07:54:50
48706 27/Mar/2013:07:21:54
48780 27/Mar/2013:07:37:57
48806 27/Mar/2013:08:09:08
48914 27/Mar/2013:07:27:39
49143 27/Mar/2013:08:12:57
49247 27/Mar/2013:07:52:17
49290 27/Mar/2013:07:10:41
49343 27/Mar/2013:07:50:01
49400 27/Mar/2013:07:37:00
49476 27/Mar/2013:07:17:13
49490 27/Mar/2013:07:18:55
49499 27/Mar/2013:07:59:02
49676 27/Mar/2013:07:26:52
49694 27/Mar/2013:07:24:52
49709 27/Mar/2013:08:01:55
49806 27/Mar/2013:07:16:26
50139 27/Mar/2013:07:51:53
50293 27/Mar/2013:07:36:54
50370 27/Mar/2013:07:57:27
50432 27/Mar/2013:07:39:01
50477 27/Mar/2013:08:06:24
50576 27/Mar/2013:07:59:43
50697 27/Mar/2013:07:23:13
50834 27/Mar/2013:08:07:47
50843 27/Mar/2013:07:32:24
51213 27/Mar/2013:07:09:25
51280 27/Mar/2013:07:58:08
51298 27/Mar/2013:07:58:15
51299 27/Mar/2013:07:55:46
51341 27/Mar/2013:07:32:37
51408 27/Mar/2013:07:51:35
51419 27/Mar/2013:07:05:13
51584 27/Mar/2013:07:24:46
51647 27/Mar/2013:07:43:44
51659 27/Mar/2013:07:19:48
51768 27/Mar/2013:08:02:58
51873 27/Mar/2013:07:06:23
52125 27/Mar/2013:08:00:58
52326 27/Mar/2013:07:16:42
52336 27/Mar/2013:07:07:16
52386 27/Mar/2013:07:10:31
52458 27/Mar/2013:08:00:00
52487 27/Mar/2013:07:26:28
52579 27/Mar/2013:07:41:03
52676 27/Mar/2013:08:06:59
52712 27/Mar/2013:07:53:37
52858 27/Mar/2013:07:39:31
52879 27/Mar/2013:07:50:48
52901 27/Mar/2013:07:20:07
52948 27/Mar/2013:07:04:13
53090 27/Mar/2013:08:10:50
53115 27/Mar/2013:07:35:58
53128 27/Mar/2013:07:58:30
53222 27/Mar/2013:07:05:54
53242 27/Mar/2013:07:06:42
53263 27/Mar/2013:07:04:47
53279 27/Mar/2013:07:56:10
53343 27/Mar/2013:07:27:45
53346 27/Mar/2013:07:56:27
53372 27/Mar/2013:08:11:08
53414 27/Mar/2013:08:05:07
53431 27/Mar/2013:07:37:35
53487 27/Mar/2013:08:06:12
53590 27/Mar/2013:07:38:32
53647 27/Mar/2013:07:33:07
53735 27/Mar/2013:07:29:36
53776 27/Mar/2013:07:05:18
53830 27/Mar/2013:07:52:49
53867 27/Mar/2013:07:26:57
54098 27/Mar/2013:07:27:31
54227 27/Mar/2013:08:04:54
54269 27/Mar/2013:08:06:19
54275 27/Mar/2013:07:20:51
54286 27/Mar/2013:07:33:15
54342 27/Mar/2013:08:12:50
54548 27/Mar/2013:08:09:03
54597 27/Mar/2013:07:29:42
54625 27/Mar/2013:07:59:33
54630 27/Mar/2013:07:27:23
54670 27/Mar/2013:08:13:03
54673 27/Mar/2013:07:07:24
54707 27/Mar/2013:08:11:44
54804 27/Mar/2013:07:55:16
55000 27/Mar/2013:07:52:41
55071 27/Mar/2013:07:54:07
55085 27/Mar/2013:07:09:00
55182 27/Mar/2013:07:27:54
55225 27/Mar/2013:07:53:47
55250 27/Mar/2013:07:09:14
55277 27/Mar/2013:07:17:57
55315 27/Mar/2013:07:21:08
55323 27/Mar/2013:07:56:58
55358 27/Mar/2013:07:28:33
55491 27/Mar/2013:07:25:00
55538 27/Mar/2013:07:31:16
55923 27/Mar/2013:07:04:55
56092 27/Mar/2013:07:42:42
56130 27/Mar/2013:07:49:54
56244 27/Mar/2013:07:19:03
56374 27/Mar/2013:07:20:59
56379 27/Mar/2013:07:36:07
56504 27/Mar/2013:07:29:16
56823 27/Mar/2013:07:18:09
56829 27/Mar/2013:07:05:46
56910 27/Mar/2013:07:30:11
56932 27/Mar/2013:07:10:06
57003 27/Mar/2013:07:43:02
57072 27/Mar/2013:07:39:53
57080 27/Mar/2013:07:10:24
57271 27/Mar/2013:07:06:50
57333 27/Mar/2013:08:08:41
57377 27/Mar/2013:07:38:53
57476 27/Mar/2013:07:18:49
57537 27/Mar/2013:08:03:28
57550 27/Mar/2013:07:17:49
58020 27/Mar/2013:07:46:47
58177 27/Mar/2013:07:41:19
58246 27/Mar/2013:07:21:44
58248 27/Mar/2013:07:08:05
58342 27/Mar/2013:07:40:07
58349 27/Mar/2013:07:40:19
58386 27/Mar/2013:07:11:33
58429 27/Mar/2013:07:07:31
58658 27/Mar/2013:07:36:29
58730 27/Mar/2013:07:18:17
58790 27/Mar/2013:07:28:14
58843 27/Mar/2013:07:37:12
58863 27/Mar/2013:08:00:32
58920 27/Mar/2013:07:38:17
58993 27/Mar/2013:08:07:06
59075 27/Mar/2013:07:39:20
59087 27/Mar/2013:08:05:52
59570 27/Mar/2013:07:14:01
59668 27/Mar/2013:07:24:37
59728 27/Mar/2013:07:03:40
59837 27/Mar/2013:07:32:08
59960 27/Mar/2013:07:14:06
59983 27/Mar/2013:07:11:08
60006 27/Mar/2013:07:04:37
60036 27/Mar/2013:07:22:12
60096 27/Mar/2013:07:49:20
60286 27/Mar/2013:08:02:19
60288 27/Mar/2013:08:05:43
60325 27/Mar/2013:07:22:53
60338 27/Mar/2013:07:30:18
60362 27/Mar/2013:07:51:20
60473 27/Mar/2013:07:29:49
60706 27/Mar/2013:07:59:12
60849 27/Mar/2013:07:07:09
60883 27/Mar/2013:08:01:12
61141 27/Mar/2013:07:39:10
61462 27/Mar/2013:07:46:28
61479 27/Mar/2013:08:09:14
61511 27/Mar/2013:07:57:46
61615 27/Mar/2013:07:05:29
61677 27/Mar/2013:07:23:55
61955 27/Mar/2013:08:12:12
62325 27/Mar/2013:07:14:19
62490 27/Mar/2013:07:04:29
62514 27/Mar/2013:07:54:21
62628 27/Mar/2013:07:36:37
62758 27/Mar/2013:07:08:31
62956 27/Mar/2013:07:33:36
63000 27/Mar/2013:07:24:21
63187 27/Mar/2013:07:30:46
63504 27/Mar/2013:08:07:56
64205 27/Mar/2013:07:23:03
64383 27/Mar/2013:08:08:48
64658 27/Mar/2013:07:45:08
64989 27/Mar/2013:07:38:06
66556 27/Mar/2013:08:06:47
67102 27/Mar/2013:07:20:13
67109 27/Mar/2013:08:12:31
67892 27/Mar/2013:07:32:57
68991 27/Mar/2013:07:16:50
69324 27/Mar/2013:07:06:31
70221 27/Mar/2013:07:59:24
70630 27/Mar/2013:07:16:34
71038 27/Mar/2013:07:19:15
71874 27/Mar/2013:07:54:37
72649 27/Mar/2013:08:10:17
73090 27/Mar/2013:07:46:10
73604 27/Mar/2013:07:23:24
75100 27/Mar/2013:07:57:15
75213 27/Mar/2013:07:28:04
75769 27/Mar/2013:07:03:47
77253 27/Mar/2013:08:10:29
79555 27/Mar/2013:07:48:14
83486 27/Mar/2013:07:15:39
87976 27/Mar/2013:07:23:47
92012 27/Mar/2013:07:04:05
94652 27/Mar/2013:07:51:03
96752 27/Mar/2013:07:20:32
97779 27/Mar/2013:07:10:48
99091 27/Mar/2013:07:31:05
100344 27/Mar/2013:07:06:59
102538 27/Mar/2013:07:51:59
103700 27/Mar/2013:07:29:01
106861 27/Mar/2013:07:05:05
115440 27/Mar/2013:07:40:33
116686 27/Mar/2013:07:18:24
120668 27/Mar/2013:07:55:54
123110 27/Mar/2013:08:09:41
124074 27/Mar/2013:07:42:30
125433 27/Mar/2013:08:09:30
127999 27/Mar/2013:07:45:28
130134 27/Mar/2013:07:43:32
131208 27/Mar/2013:07:08:51
131579 27/Mar/2013:07:43:29
132537 27/Mar/2013:07:06:04
133762 27/Mar/2013:07:19:33
137204 27/Mar/2013:07:53:02
137436 27/Mar/2013:07:18:34
138092 27/Mar/2013:07:41:32
138458 27/Mar/2013:08:08:28
141186 27/Mar/2013:07:27:15
144419 27/Mar/2013:07:35:52
145903 27/Mar/2013:07:31:56
146732 27/Mar/2013:07:52:09
148267 27/Mar/2013:07:17:31
150219 27/Mar/2013:07:58:53
154578 27/Mar/2013:07:57:55
158174 27/Mar/2013:07:38:45
165821 27/Mar/2013:08:12:21
166931 27/Mar/2013:07:25:56
169495 27/Mar/2013:07:35:45
170584 27/Mar/2013:07:32:50
179257 27/Mar/2013:07:09:50
182595 27/Mar/2013:08:04:43
189441 27/Mar/2013:08:04:35
189498 27/Mar/2013:07:03:58
189830 27/Mar/2013:07:24:12
200822 27/Mar/2013:07:30:04
206895 27/Mar/2013:07:37:48
207304 27/Mar/2013:08:03:41
212494 27/Mar/2013:07:05:39
214665 27/Mar/2013:07:25:15
230095 27/Mar/2013:08:11:25
230242 27/Mar/2013:07:51:08
231758 27/Mar/2013:07:26:18
234830 27/Mar/2013:07:37:41
236487 27/Mar/2013:08:13:16
241560 27/Mar/2013:07:19:27
242001 27/Mar/2013:07:38:39
247939 27/Mar/2013:07:39:42
250885 27/Mar/2013:08:02:38
262732 27/Mar/2013:07:03:09
268596 27/Mar/2013:07:33:47
356092 27/Mar/2013:07:21:20
364474 27/Mar/2013:07:21:34
427457 27/Mar/2013:07:22:19
430091 27/Mar/2013:07:22:41
471204 27/Mar/2013:07:11:00
565813 27/Mar/2013:07:26:10
600867 27/Mar/2013:07:08:19
665232 27/Mar/2013:08:00:42
676467 27/Mar/2013:07:20:22
698953 27/Mar/2013:07:30:54
740642 27/Mar/2013:07:11:41
838301 27/Mar/2013:07:27:05
858092 27/Mar/2013:08:09:21
869306 27/Mar/2013:07:13:50
963469 27/Mar/2013:07:07:51
972483 27/Mar/2013:07:07:41
1437553 27/Mar/2013:08:12:40

For me, its happening only at fight page, but while fight page lagging, it doesnt load any other bots4 page as well. Never had any lags, in other pages which wouldnt be caused from my side. Lag freeze could take from few seconds to minute or more. Looks like server responding, fight is loaded and happening in that time, cos after lags end usually its louding fight result. Lag could take few fights and after it everything fixed again. In my case, i dont get any errors.

I think first time it happen last week or something around it. Since then, it happens every 2-3 days

1. when it happened to me, all the pages of bots4 took very long to load. I dont remember exactly if other pages, like google, were normal or not.

2. more than one second for sure. Long enough to mess with my nervous system while I was dumping.

3. well, this I dont remember very well. Game errors only the one when I double click fight again button.

4. If happen again, I'll memerize the hour.

For it was:

1: all pages, even when logged out and pressing home would lag, only occured on bots4, all other sites fine, plus have tools running for work.

2: lag from 30 sec up too 2-3 minutes

3: no error messages at all

4: the moment it happened i posted on bots4 facebook page as i thought it was a server crash or something.

PS during the lag sessions :) i actually pinged www.bots4.net and they came back fine

Thanks for the info. This is tricky. It could be any number of things, but without a lot of profiling it's probably going to be hard to find. Incoming braindump:

I'm tempted to increase the size of the VPS the game runs on, but this will effectively ~double my operating costs. Plus it might not even fix the problem if, for example, MySQL or Apache needs to be tuned better or something.

Another option would be to move the database to a separate server, which might alleviate some of the load. This would also increase operating costs, but having a second server would allow me to put up a maintenance page when either goes down for maintenance (which is something I pretty much don't do now, which is bad).

Then again, this sounds like the problem is pretty intermittent, so I'm also tempted to not address it right now. It's complicated, hard to fix, and from what I can tell doesn't affect too many people too often.

But let's start with something simple. :) The server hasn't been rebooted in over a year and a half (!), so let me try that. I'll try to do this sometime tonight.

I rebooted the server just now. Let me know if you still see issues.

How have things been since the reboot?

Yesterday I didnt have problems...

So far so good for me too.

Nice, I'll have to restart the server more often. :)

thanks ender since you did this i haven't experienced any problems

did anyone just experience a very bad lag?

3:57:02 time what it happened at ender

5:17 42 am , very bad lag again :/ , - Lobster

I checked the logs for those times and don't see anything out of the ordinary. Server response time is in the 25-250ms range, so this is sounding like a network issue. Not sure what I can do about it, if anything, but I'll keep this in mind.

i can't see why its doing this i checked with other sites before posting :/ i just cant see why it would be my network when i loaded a few other sites up :/

Isn´t it that time of the year again? Day-light saving or something like that?

It is. Yesterday started summer time.

Network issues in general, not necessarily on your end.

This isn't related to DST.

6:12:35 am by the page eventually responded it was 6:15:39 am

its only happening to bots though why?

3+ hours me or my brother haven't been able to get on bots but again everything else was working any tips would be appriciated because im pissed off right this moment

Is it not something locally leader? No one else is having the same issue, but you and your brother are

im honestly not sure whats happening what ever happened today it prevented me from playing bots :(

you should try to trace bots4.net when lag is happening. for win systems its 'tracert bots4.net' in console. there you can see if some point gives notable higher ping than others (like hops 8-10 runs around 150ms and 11th takes 1000ms <- 11th point is lagging alot).

i havent noticed any lag at all...

A 3+ hour outage is something the server monitoring would have picked up. I would have heard about it from others too. This is starting to sound like a network issue either on your end or somewhere between you and the server. Not sure there's anything I can do to fix this, let alone debug it.

What would you advise i do then? i was using firefox if thats helps?

The server is fine, im sure of that. it has to do something with hops between user <-> server.

I got some random lag problems when using Firefox but none when using Chromium/Chrome.

Has anyone else been getting some long lag spikes just now? Sites other than bots4 are loading fine for me, but at times it is taking 5+ mins to load bots4 pages.

over the past hour, i've experienced some lag on multiple bots while loading other websites fine

same here. i thought it was just me so i disconnected for a bit.

Ive noticed it every now and then

Mass Psychogenic Illness
Hysterical Contagion
Conversion Syndrome

Perhaps instead of investing in pingdom monitoring you should invest in an ECT machine Ender. If you feel such a machine is too expensive, a car battery ought to do and using it to give the above posters a little 'therapy' will negate the need for this thread.

wonder if there is an IPv4 bottleneck on the network somewhere close to Enders server, since so many people from different places around the world seems to have the same lag issues, that apparently is not related to any server lag ...

had major lags yesterday too

was wondering if Heap Snapshots or Javascript CPU Profioles might be any help? or anything else that could be monitored by Chrome on the client side