00:02 < bridge> what do you mean by "entire thing"? 00:03 < bridge> the entire translation .txt, doesnt seem profitable to make a PR for a single translation :P 00:05 < bridge> why are you translating Tee again 00:05 < bridge> the players know what tees are 00:05 < bridge> just like they know what DDNet is 00:05 < bridge> Because I found the term that is used on Wiki 00:05 < bridge> i don't necessarily think this is the correct way to do that 00:06 < bridge> Having "тії" is better than having "тее", change my mind 00:07 < bridge> Tee is a identity, not something you translate 00:08 < bridge> translating `Tee` seems fine to me, but i'd consider `Tee` to be a proper noun, wouldnt really translate that 00:08 < bridge> but i'd consider `Tee` to be a proper noun, wouldnt really translate that 00:08 < bridge> "Combine" is also a proper noun, but it didn't stop translators 00:09 < bridge> that’s because it’s critical to the HL2 plot and wouldn’t make sense otherwise 00:09 < bridge> Slavic languages love transliterating proper nouns. Макдоналдс 00:09 < bridge> ^ 00:10 < bridge> i don’t think that makes it correct 00:10 < bridge> Старбакс 00:10 < bridge> harry potter is also transliterated in russian 00:10 < bridge> also, um, it is not finished yet. Again, 10 strings, and I don't know where they occur, meaning I can't really make an accurate translation 00:11 < bridge> And they use a г so it’s garry potter 00:11 < bridge> I wouldn't say it's garry potter 00:11 < bridge> are these the missing ones? 00:11 < bridge> just pronounced as such 00:11 < bridge> yes, those are the ones I haven't translated yet 00:13 < bridge> Well if a russian person only speaks russian, for all they know it might be garry everywhere 00:13 < bridge> I guess 00:13 < bridge> but it's also gitler, so it's a consistent transcription, at least 00:14 < bridge> I do wonder why leading h becomes g in russian though 00:14 < bridge> They do have the sounds for it х sounds much closer to my ear atleast 00:15 < bridge> probaly cuz russian developed without the "h" sound 00:15 < bridge> probaly cuz russian language developed without the "h" sound 00:20 < bridge> literally 10 strings (or 13, if including PRs, who knows), and the translation is done 00:21 < bridge> btw, should I try again adding context to "Name" in Server browser? 00:22 < bridge> oh wait it hasn't been fixed yet 00:23 < bridge> hasn't been fixed yet 00:23 < bridge> can someone try and reproduce crashing the client by rendering a demo, setting the playback time to 0.1 - forwarding a bit and then quitting out of it without finishing rendering using the X on the bottom right? - my client became unresponsive for a solid minute, outputted me this: 00:23 < bridge> 00:23 < bridge> ``` 00:23 < bridge> 2024-06-04 00:18:38 I demo_player: Stopped playback 00:23 < bridge> 2024-06-04 00:18:38 I demo_player: Loading demo 'demos/replays/asdasd.demo' 00:23 < bridge> 2024-06-04 00:18:38 I datafile: could not open 'maps/Adrenaline 3.map' 00:23 < bridge> 2024-06-04 00:18:38 I videorecorder: Recording to 'videos/asdasd.mp4.mp4' 00:23 < bridge> 2024-06-04 00:18:59 I client: disconnecting. reason='unknown' 00:23 < bridge> 2024-06-04 00:18:59 I video_recorder: ------------ 00:23 < bridge> 2024-06-04 00:18:59 I video_recorder: ------------ 00:23 < bridge> 2024-06-04 00:18:59 I demo_player: Stopped playback 00:23 < bridge> ``` 00:24 < bridge> 00:24 < bridge> and then closed (using DDNet 18.2) 00:24 < bridge> 00:24 < bridge> this is the output on latest upstream: 00:24 < bridge> 00:24 < bridge> ``` 00:24 < bridge> 2024-06-04 00:22:47 W videorecorder/libav: Too many bits 32768.000000 > 12288 per frame requested, clamping to max 00:24 < bridge> 00:24 < bridge> 2024-06-04 00:22:47 I videorecorder: Recording to 'videos/aseasdasdasd.mp4' 00:24 < bridge> 00:24 < bridge> 2024-06-04 00:22:51 I videorecorder/libav: frame I:114 Avg QP:14.91 size:177571 00:24 < bridge> 00:24 < bridge> 2024-06-04 00:22:51 I videorecorder/libav: frame P:412 Avg QP:24.00 size: 10011 00:24 < bridge> 00:24 < bridge> 2024-06-04 00:22:51 I videorecorder/libav: frame B:831 Avg QP:25.97 size: 3511 00:24 < bridge> 00:24 < bridge> 2024-06-04 00:22:51 I videorecorder/libav: consecutive B-frames: 12.7% 9.6% 22.3% 55.4% 00:24 < bridge> 00:24 < bridge> 2024-06-04 00:22:51 I videorecorder/libav: mb I I16..4: 32.8% 25.2% 41.9% 00:24 < bridge> dont know if its my fault or actual issue - so i didnt immediatly make one 00:26 < bridge> waiting then 00:26 < bridge> as well as for \#8404 00:28 < bridge> changing `Spectate` to `Spectating` means you have to change the string in every translation.txt file, so the PR is incomplete it seems? 00:28 < bridge> nvm deen already said something about it 00:28 < bridge> no, that cna be handled differently 00:58 < bridge> https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html 00:58 < bridge> This feels very overdue 03:47 < bridge> maybe so but everyone i know who seriously uses rust on linux is linking with mold 03:48 < bridge> True, I'm using mold as well 08:29 < bridge> @nuborn @jupeyy_keks do have you got any good ideas for a net message struct that could replace NETMSG_INPUT when sending more than 1 input at once to save size? 08:29 < bridge> 08:29 < bridge> It seems that nearly 100% of the client's upload bandwidth is already from sending NETMSG_INPUT which makes 8441 pretty expensive in bandwidth. I was able to reduce it from 800 inputs per second maximum to 250 inputs per second maximum by assuming a minimum of 1/4 packets reach the server which gets it to reasonable bandwidth levels but it doesn't perform nearly as well above even 30% packet loss which makes me sad. 08:29 < bridge> 08:29 < bridge> Currently the client sends about 0.5KB/s-1.75KB/s of NETMSG_INPUT (without dummy), after the PR it's about 6KB-12KB/s. If I limit it to 250 inputs/s it gets down to 2.5KB/s-7KB/s which is more reasonable. The compression ratio on these seems to be around 1.5-1.7:1.0 08:29 < bridge> 08:29 < bridge> Other real time games seem to be in the 4-25KB/s range like CSGO/Overwatch/Valorant/Fornite. Those games of course probably already send duplicated inputs, but don't have a way for clients to manually adjust the "prediction margin", it would just be adjusted automatically based on network health so it would be much lower on average. DDNet client also attempts to adjust automatically but seems to fail very bad even in any scenario I tested, even with 08:29 < bridge> 08:29 < bridge> For the most part there's only 4 pieces of info that change in every individual NETMSG_INPUT which is m_AckGameTick, m_PredTick, m_TargetX, m_TargetY. The other things are limited to your human ability to press a button quickly which is pretty slow. 08:29 < bridge> 08:30 < bridge> It would be pretty easy to send m_AckGameTick, and m_PredTick only once for the entire message then increment them manually on the server for each contained input but not as easy for m_TargetX, m_TargetY. 08:30 < bridge> 08:30 < bridge> I have some doubts about how well a naive "binary diff" will work like Juppey suggested, it doesn't seem viable for such tiny sizes. ChatGPT suggested bit packing all the button presses into single bits then sending mouse deltas instead of mouse positions after the initial one, but I'm not sure that helps since the deltas also need to be full ints I think. Also we have m_Fire which is hogging 8 bits minimum. 08:32 < bridge> <0xdeen> > Currently the client sends about 0.5KB/s-1.75KB/s of NETMSG_INPUT (without dummy), after the PR it's about 6KB-12KB/s. If I limit it to 250 inputs/s it gets down to 2.5KB/s-7KB/s which is more reasonable. The compression ratio on these seems to be around 1.5-1.7:1.0 08:32 < bridge> <0xdeen> I'm afraid our servers might get overloaded with that 08:33 < bridge> well that's why I'm writing this message 08:33 < bridge> alternative would be just set a limit and accept whatever packet loss happens after that 08:34 < bridge> i dont understand why binary patching should not be the solution, isnt the input mostly the same for almost always? 08:35 < bridge> I don't understand how binary patching can define the patch positiona and the patch data in less bits than bitpacking+huffman 08:35 < bridge> I don't understand how binary patching can define the patch position and the patch data in less bits than bitpacking+huffman 08:35 < bridge> people on stackoverflow say it shouldn't be used like this 08:36 < bridge> but the bitpacking and huffman can be applied either way 08:36 < bridge> actually compression even must be applied 08:36 < bridge> surely there can be some solution though as this input duplication was invented by quakeworld in 1996 with like 1% of our current bandwidth 08:37 < bridge> surely there can be some solution though as this input duplication idea was invented by quakeworld in 1996 with like 1% of our current bandwidth 08:38 < bridge> give me the raw data for an input chain and we can compare which method wins 08:38 < bridge> i can write the binary patch part very quickly for it 08:38 < bridge> https://stackoverflow.com/questions/48871467/binary-diff-algorithm-for-small-binary-blobs 08:39 < bridge> idk 08:39 < bridge> well you have nothing to do, i'll do it 08:39 < bridge> wdym 08:40 < bridge> just give me the binary data of your packets 08:40 < bridge> and i'll try some stuff 08:40 < bridge> forgot about this one: https://github.com/ddnet/ddnet/pull/8423 08:40 < bridge> it only sends 1 packet per tick 08:40 < bridge> but it sends multiple packet chunks or whatever 08:41 < bridge> or just give me the 40 bytes of the inputs 08:41 < bridge> then tell me how tall that packet is on ddnet 08:41 < bridge> and i try to beat it 08:41 < bridge> it's not that simple 08:42 < bridge> give me the packets before the packer struct is used 08:42 < bridge> ok but what do you mean the packets 08:42 < bridge> before or after huffman 08:42 < bridge> before 08:42 < bridge> the raw input data basically, but ofc all 20 or how many u have 08:43 < bridge> ok 08:43 < bridge> then ddnet sends a single packet with all.. this size is interesting bcs that's what to beat (minus the packet header) 08:44 < bridge> why don't you just clone my PR and write it manually then measure the bandwidth used 08:44 < bridge> you can see outgoing bandwidth even if the server is ignoring your garbage packet data 08:44 < bridge> but i dunno what u did to measure this 08:45 < bridge> just system bandwidth usage 08:45 < bridge> I used some windows program utility 08:45 < bridge> https://www.netlimiter.com/ 08:45 < bridge> I used some windows utility program 08:46 < bridge> NETMSG_INPUT is like 99% of the realtime client upload bandwidth anyway so this is still accurate 08:47 < bridge> just system process bandwidth usage 08:50 < bridge> morning 08:56 < bridge> What issue does this 12x increase in traffic solve? 08:56 < bridge> <0xdeen> Sure, more expensive servers 08:57 < bridge> Or less players per server 08:58 < bridge> view at timestamp 08:58 < bridge> https://www.youtube.com/watch?v=W3aieHjyNvw&t=1891s 08:58 < bridge> https://private-user-images.githubusercontent.com/22122579/335931179-8c02bd20-fdc5-4d34-80d4-0a47ca8369da.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTc0ODQ1NjUsIm5iZiI6MTcxNzQ4NDI2NSwicGF0aCI6Ii8yMjEyMjU3OS8zMzU5MzExNzktOGMwMmJkMjAtZmRjNS00ZDM0LTgwZDQtMGE0N2NhODM2OWRhLm1wND9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09 08:58 < bridge> ok discord didn't like that 08:59 < bridge> There is a demo here: 08:59 < bridge> https://github.com/ddnet/ddnet/pull/8441 08:59 < bridge> <0xdeen> Isn't this dangerous? It means we trust the client more now and the client can manipulate the past? I'm wondering if this can give attackers an advantage to cheat 08:59 < bridge> no 09:00 < bridge> client cannot change inputs that it already sent to server after https://github.com/ddnet/ddnet/pull/8432 09:00 < bridge> no extra trust 09:01 < bridge> even if, it can only manipulate queued inputs.. so there isn't really an advantage that isnt there yet.. 09:01 < bridge> 09:01 < bridge> it could not change past inputs (in the sense that the logic already processed them) 09:02 < bridge> it has danger after #8372 09:02 < bridge> https://github.com/ddnet/ddnet/pull/8372 09:02 < bridge> but ddnet is not PvP so really it doesn't matter 09:02 < bridge> the client could already keep a local buffer of inputs if it really wanted 09:02 < bridge> you could play the entire map offline then play it back as a replay bot 09:03 < bridge> but even here, if one player does not use prediction margins, the other player doesnt really gain anything 09:03 < bridge> in PvP you could do something like trick your opponent into thinking you're in 1 position then you change your inputs on the server so you're actually in another position 09:03 < bridge> CS:GO has this vunerability and it is in cheats 09:04 < bridge> because they let the client modify inputs it already sent to the server buffer 09:04 < bridge> for some reason 09:04 < bridge> CS:GO had this vunerability and it is in cheats 09:05 < bridge> but still.. this only works if u use a reasonable high inputs queue in first place 09:05 < bridge> a _good_ client doesn't need a high prediction margin 09:05 < bridge> a cheating client doesn't care 09:05 < bridge> You can use less bytes for targetx and targety if their deltas are bounded. Similarly for ackgametick and predtick, you can only send deltas which are bounded 09:06 < bridge> so the cheating client sets a high prediction margin with low latency network basically? 09:06 < bridge> 09:06 < bridge> mh yeah that could work 09:06 < bridge> bounding the delta of targetx targety would be a change in client capabilities, is that ok? 09:06 < bridge> idk how you even enforce that 09:07 < bridge> Is it even physically possible to go faster than like 2 bytes at a time 09:07 < bridge> sometimes it is 09:07 < bridge> t0 player has spoken 09:07 < bridge> targetx targety is scaled by zoom so you can move abritrarily fast in the client already 09:07 < bridge> hmm, what about pvp mod hosters? 09:07 < bridge> not often 09:08 < bridge> Ah, that's a shame 09:08 < bridge> 🌞 09:08 < bridge> they will not receive this packet 09:08 < bridge> ddnet only 09:08 < bridge> Perhaps a magic delta value that indicates the next targetx and targety are not delta but full 09:08 < bridge> 0xFFFF 09:08 < bridge> also as I said there is no risk because the client cannot change values it already sent 09:09 < bridge> I see 09:10 < bridge> The rest of the input packet should compress very well though, do you do that yet? 09:10 < bridge> the compression performance of all packets is rather poor because the huffman table is 15 years old I think 09:10 < bridge> I only got like 1.5:1 09:11 < bridge> I would be quite surprised if that is an actual issue 09:11 < bridge> idk 09:11 < bridge> perhaps not 09:11 < bridge> 1.5:1 is not great either way 09:12 < bridge> But why rely on huffman anyway when we know the pattern already, the most common thing that will happen is that the input won't change. That's 1 bit you need to sneak in somewhere 09:13 < bridge> yes that's true, we can do better than huffman 09:13 < bridge> Why was huffman chosen in the first place? I think the nature of this kind of data makes it hard to compress 09:13 < bridge> huffman is just bonus 09:13 < bridge> ask magnus 09:13 < bridge> although I don't think it's a terrible choice 09:14 < bridge> it does better than nothing 09:14 < bridge> Exactly anything our naive but effective bit cant help with. Huffman can get a bit better 09:15 < bridge> there are instances where manually minimizing your data actually increases the overall size after compression because it has more entropy at the compressor, but I doubt this will be an issue for us because our huffman is very weak. 09:16 < bridge> turning duplicate bits into smaller non-duplicate bits makes compression worse 09:16 < bridge> sometimes 09:16 < bridge> Wherever structured data is involved, huffman coding usually works well-ish. It is also rather performant compared to alternatives 09:16 < bridge> What would you use? 09:17 < bridge> Can I get a short TL;DR why we are talking about minimizing data? 09:18 < bridge> we would like to sent duplicated input packets to minimize the lag that happens when packet loss occurs, but the total bandwidth is already 99% input packets so its important we minimize their size 09:18 < bridge> He wants to resend inputs that havent been acknowledged by the server yet 09:18 < bridge> we would like to send duplicated input packets to minimize the lag that happens when packet loss occurs, but the total bandwidth is already 99% input packets so its important we minimize their size 09:19 < bridge> Oh I see, have you tried it already? 09:19 < bridge> Or is everything theoretical? 09:19 < bridge> demo in here 09:19 < bridge> 09:19 < bridge> possibly this would also make DDoS less effective if we can make the inputs small enough 09:20 < bridge> Only a meager 12x increase in traffic 😄 09:20 < bridge> i doubt that, but it makes it less horrible for smaller lags 09:20 < bridge> Damn 09:20 < bridge> 12x is worst case if client is spamming their keyboard 09:20 < bridge> also it assumes you actively use prediction margins 09:20 < bridge> in theory the client should automatically adjust prediciton margin based on packet loss but this is a future issue 09:21 < bridge> yes but then the ddos pattern simply changes to 2 second massive attacks 09:21 < bridge> and then it doesnt work either wa 09:21 < bridge> and then it doesnt work either way 09:21 < bridge> Or worse because now our clients are also effectively ddosing us with inputs :thinkies: 09:21 < bridge> I doubt it 09:21 < bridge> How big is the rest of the input packet btw? 09:22 < bridge> wdym rest 09:22 < bridge> Except the 4 fields we already discussed 09:22 < bridge> 40bytes for input + the 12 bytes for 3 extra info ints 09:23 < bridge> What are in the extra info ints? Also slow changing stuff? 09:23 < bridge> m_AckGameTick, m_PredTick, m_Size 09:23 < bridge> m_Size is actually fixed at 40 and could maybe be deleted? 09:24 < bridge> its only incase we increase the size of the input struct 09:24 < bridge> I assume? 09:24 < bridge> Idk why it exists, been years since I last looked into input handling 09:24 < bridge> much of it is bad so it would not surprise me if we can remove it 09:25 < bridge> the client and server sent the input struct as abstract data which gets copied back into the actual struct later 09:25 < bridge> also I think the chance of use changing the input struct is pretty low 09:25 < bridge> also I think the chance of us changing the input struct is pretty low 09:27 < bridge> If we chunk the input packet into bytes (or twobytes). We can have two (or one) byte(s) whose bits can track whether their respective bytes (or twobytes) have changed. That would compress 40 bytes to 2-1 bytes in the common case that input doesn't change 09:27 < bridge> I would be interested in game tick negotiation, so we could run 100 tick servers instead of 50 09:27 < bridge> if DDoS is killing server performance then duplicate inputs would not help/make it worse, but if the DDoS is clogging network bandwidth then it would help a lot 09:27 < bridge> but i guess this will kill the physics, no? they are heavily coupled to the ticks 09:27 < bridge> this changes physics too much I think 09:27 < bridge> there was discussion and several prototypes on github 09:28 < bridge> <0xdeen> Absolutely 09:28 < bridge> it makes gores way easier :) 09:28 < bridge> Yeah 100ticks is a no go I think 09:28 < bridge> <0xdeen> You could have a way to allow it for new maps, but it might feel strange for players used to 50 ticks 09:29 < bridge> for gores it could work 09:29 < bridge> since it's not as bound to weird physics hacks 😄 09:29 < bridge> Anyone who implements it better have very concrete proof that if set at 50hz it still behaves the exact same as it used to 09:29 < bridge> we use sv_high_bandwidth on blockworlds, it might slightly change the physics but cosmetics are way smoother when it's enabled? 09:30 < bridge> we use sv_high_bandwidth on blockworlds, it might slightly change the physics but cosmetics are way smoother when it's enabled 09:30 < bridge> it does not change physics, just increases snapshot rate to clients 09:30 < bridge> 25->50 snaps/s 09:30 < bridge> Interesting 09:30 < bridge> When neural network based prediction? 09:30 < bridge> would be nice to have on offical servers 09:31 < bridge> Mhmmm, never tried before, but it could 09:31 < bridge> We would have to check and see how much more bandwidth it would use. Snapshots are delta compressed so it shouldnt be too too bad 09:31 < bridge> But I'd be in favor of negotiation in first place 😄 Not just setting the server to 100 ticks 09:31 < bridge> I assume its near double 09:31 < bridge> so instead of 50 ticks/sec it's 100 ticks/sec 09:32 < bridge> I would guess about 1.5x 09:32 < bridge> NO 09:32 < bridge> The state is snapshot more often. The state doesn't mutate more often. The tickrate refers to how quickly the state mutates 09:33 < bridge> it depends just because they are delta compressed doesn't mean it won't double it. Only if full snapshots stay the same and delta snapshots increase would it have less than 2x increase 09:33 < bridge> it depends, just because they are delta compressed doesn't mean it won't double it. Only if full snapshots stay the same and delta snapshots increase would it have less than 2x increase 09:33 < bridge> but if most of the bandwidth is already deltas then it will 2x 09:33 < bridge> since you can't get smaller than that 09:33 < bridge> Yeah would have to observe 09:33 < bridge> oh okay 09:34 < bridge> also it affects demo size 09:34 < bridge> on client PCs 09:34 < bridge> so all input packets are always the same size? 09:34 < bridge> can i assume that? 09:34 < bridge> yes 09:34 < bridge> nice 09:34 < bridge> that makes it even easier 09:34 < bridge> It appears more smooth because you get a snapshot about every tick. Instead of the default every other tick 09:34 < bridge> I guess this is true 09:35 < bridge> idk how much that helps but it does 09:35 < bridge> if sending more deltas means you send less deltas per tick it could be lower 09:35 < bridge> if sending more snaps means you send less deltas per tick it could be lower 09:36 < bridge> I was referring to Noa's message still 🙃 09:36 < bridge> ah right 09:36 < bridge> unintentially interesting idea tho xd 09:38 < bridge> Anyway, idk. I don't think we can guesstimate much since I actually don't remember how much of the traffic is deltas, but the sent deltas may or may not get smaller depending on whether we have things that change slower than 50hz faster than 25hz 09:39 < bridge> I think it should also reduce average latency by 20ms right? 09:40 < bridge> It should reduce it but I don't know why it would be by a flat number 09:40 < bridge> well on average 09:40 < bridge> at any given point in time there is between 0-40ms until the next snap the server sends you on 25tps 09:40 < bridge> at 50tps it's 0-20ms until the next one 09:41 < bridge> Ah, I see what you mean. Yeah that sounds about reasonable 09:43 < bridge> <0xfaulty> Increasing the speed of sending inputs would probably increase the accuracy of the antibot (at least my implementation), as often the change of inputs looks extremely strange, but in fact the reason for this is packet delays 09:43 < bridge> you're using client side antibot? 09:43 < bridge> <0xfaulty> server side ofc 09:43 < bridge> the server already recieves the full 50 inputs per second for every player. There can't be more than that unless you change the physics 09:44 < bridge> only clients are limited to 25hz 09:44 < bridge> only clients are limited to 25hz (for information about other players) 09:44 < bridge> <0xfaulty> This is in the perfect case, but in reality there are delays 09:44 < bridge> I guess sending it quicker wouldn't technically change the physics per se, but the feel 😄 09:45 < bridge> I'm not sure what you mean 09:45 < bridge> client side prediction ensures the feel is the same regardless of how fast your input reaches the server 09:45 < bridge> @totar so i have the binary diff now.. if u can now tell me how i can manually call compress from cpp code i can check how much better binary patching performs if i use ddnet's huffman compression 09:46 < bridge> @totar 's new patch would mean input packets would be less likely to be lost. Which would make your antibot get a more continuous appearing data stream. But you should probably be correcting for random transmission delays as those convey no information really 09:48 < bridge> I don't have some sort of compression testing framework for you to easily call a single function on the input data. You have to pack all the data for each input message (the 3 ints + player_input) into a single segment of memory, then add each segment of it to the message as an int. 09:48 < bridge> ok i'll just create a fake packet using the packer thing 09:48 < bridge> lets see if that works 09:48 < bridge> yeah exactly 09:48 < bridge> the packet can be garbage as far as the server knows 09:49 < bridge> you just need to make sure that whatever you put into the message can theoretically be converted back into the full data on the other side with all the information you give it 09:50 < bridge> <0xfaulty> You right, I meant continuity exactly, delays could be handled but missed data not 09:50 < bridge> this guy on reddit had an interesting idea 09:50 < bridge> https://cdn.discordapp.com/attachments/293493549758939136/1247457548127305779/image.png?ex=666018e0&is=665ec760&hm=5184633a944dbb3494b376f9fbd019b63b16e70ca8f1f3981afea5bcc03637ea& 09:50 < bridge> yeah since the input packets are same size that's super ez xd 09:51 < bridge> if you turn all the data into 0s then the generic compressor should make it really small 09:52 < bridge> There are many ways to compress slowly changing data 09:52 < bridge> less ways when it's only 52 bytes 09:54 < bridge> Well you can do exactly what he says and it would work fine, but I wouldn't really use something like lz4. Just rle 09:55 < bridge> Or you can do what I suggested with one or two bytes tracking the rest of the bytes 09:57 < bridge> I'm still a little dissapointed by the 1.5:1 by the huffman. like 80% of the input struct should always be zeros without any special tricks 09:57 < bridge> https://cdn.discordapp.com/attachments/293493549758939136/1247459146069053470/image.png?ex=66601a5d&is=665ec8dd&hm=6add5a0c3e4657f00d09809898a22610a26a232e1d27af756353807e9883ca52& 09:57 < bridge> I'm still a little dissapointed with the 1.5:1 by the huffman. like 80% of the input struct should always be zeros without any special tricks 09:57 < bridge> https://cdn.discordapp.com/attachments/293493549758939136/1247459146069053470/image.png?ex=66601a5d&is=665ec8dd&hm=6add5a0c3e4657f00d09809898a22610a26a232e1d27af756353807e9883ca52& 09:58 < bridge> so how can i identify my packet in wireshark now lmao 09:58 < bridge> wireshark is hard sorry lol 09:59 < bridge> look up some tutorials 09:59 < bridge> chiller would know it 09:59 < bridge> true 09:59 < bridge> Well the table is not for just inputs (I don't even know what the table is generated by/for). Huffman doesn't perform that well without seeing a lot of input 10:00 < bridge> I don't think the data to be comrpessed matters if the huffman table is fixed? 10:00 < bridge> I don't think the size of data to be comrpessed matters if the huffman table is fixed? 10:01 < bridge> We could have a huffman tree just for inputs and that might do better. It's have lots of sequences of zeros on short codes 10:01 < bridge> @totar i think the constant 136 bytes are my packet 10:01 < bridge> https://cdn.discordapp.com/attachments/293493549758939136/1247460095370199150/image.png?ex=66601b3f&is=665ec9bf&hm=461493dd51707345f9a07a67d8939a7d4ef5d48df0153bd9d2067fa666a2df3c& 10:01 < bridge> and the bigger ones are the normal resent packets 10:01 < bridge> I do not know how to use wireshark idk what you want from me lol 10:01 < bridge> 136bytes for 15 inputs? 10:01 < bridge> 13 inputs 10:01 < bridge> ok 10:01 < bridge> quite good I think right? 10:01 < bridge> i guess so 10:02 < bridge> hmm 10:02 < bridge> using brotli compression i get exactly the same lol 10:02 < bridge> actually that's 6.8KB/s 10:02 < bridge> I think 10:03 < bridge> which is still better than what I had at 13 inputs probably 10:03 < bridge> ah but it would loose if i'd add packet headers rip 10:03 < bridge> are you spinning your mouse in circles? 10:03 < bridge> that matters a lot 10:03 < bridge> yes 10:03 < bridge> ok 10:03 < bridge> i tried to jump and spin 10:03 < bridge> and move 10:03 < bridge> and press alt + f4 at the same time 10:03 < bridge> lmao 10:03 < bridge> lol 10:03 < bridge> Rq 10:04 < bridge> But how fast is yours vs brotli? 10:04 < bridge> the brotli part is only interesting bcs i wrote it in rust 10:04 < bridge> was just a test 10:05 < bridge> A lot of the big math smart guy optimizations don't kick in until sizes much larger than a couple hundred bytes 😛 10:05 < bridge> that's correct 10:05 < bridge> but i think for a general purpose compression that's still pretty good 10:05 < bridge> Yeah I think that's pretty good. How did you end up implementing it? 10:06 < bridge> also brotli is not designed for binary data actually lmao 10:06 < bridge> 136 bytes is about 5:1 which is the same as the amount of zeros to data in the orignal struct that I calculated a few minutes ago 10:06 < bridge> the binary patch? 10:06 < bridge> Yes 10:06 < bridge> very straight forward actually 10:06 < bridge> ```rust 10:06 < bridge> let mut writer = Vec::new(); 10:06 < bridge> let empty = vec![ 10:07 < bridge> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10:07 < bridge> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10:07 < bridge> ]; 10:07 < bridge> let mut cur_last = ∅ 10:07 < bridge> for a in a_bytearr.iter() { 10:07 < bridge> fn diff(old: &[u8], new: &[u8], writer: &mut Vec) { 10:07 < bridge> for (index, old) in old.iter().enumerate() { 10:07 < bridge> writer.push(new[index].wrapping_sub(*old)); 10:07 < bridge> } 10:07 < bridge> } 10:07 < bridge> diff(cur_last, a, &mut writer); 10:07 < bridge> cur_last = a; 10:07 < bridge> } 10:07 < bridge> ``` 10:07 < bridge> a_bytearr is simply the i32 converted to u8 10:07 < bridge> so for all input packets i simply create the binary diff towards the previous one 10:09 < bridge> I think I need my coffee and breakfast now. That looks like it encodes 40 bytes to 40 bytes diff 10:09 < bridge> it simply takes the difference 10:09 < bridge> thus creating lots of 0s 10:09 < bridge> that's the whole point of it 10:10 < bridge> and if not 0s then at least some smaller ints 10:10 < bridge> Ah, so you just let huffman coding take care of it 10:10 < bridge> yeah compression is applied nonethenless 10:11 < bridge> i basically just want to help the compressor a bit 10:11 < bridge> since inputs are always same size and always similar values 10:11 < bridge> I guess the longer sections of 0 are easier on the huffman table we have for some reason 10:11 < bridge> couldn't you just xor the previous input with the next one? 10:11 < bridge> like the guy on reddit said 10:12 < bridge> You can 10:12 < bridge> well nice 10:12 < bridge> but is xor even better? well i dunno 10:12 < bridge> my math brain is off rn 10:12 < bridge> I can't imagine how much it changes rn. Not before breakfast 10:12 < bridge> is this not equivalent to xor? 10:13 < bridge> if both are 40bytes the difference is actually just the xor? 10:13 < bridge>