Rendered at 15:01:08 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
setheron 11 hours ago [-]
(author) -- fun to see this here.
I wrote this on the way (on the airplane) to DEFCON34 and Nix Vegas[1]
I didn't even get a chance to cruise around and see anything of DEFCON proper since the Nix community had me captive the whole time!
I gave 3 talks that I will update on my site[2] once they released.
Neat! Very sad to have missed Nix Vegas, sounds like it was particularly good this year. I[0] hacked together something similar[1] for the Game Boy a couple months ago[2], outputting frames with builtins.trace instead of images, and it ended up being useful as a memory pressure test while working on a Nix evaluator implementation[3].
Pokemon was so slow that coming up with TAS scripts, even with state saves/loads for every run, became infeasible, so I ended up searching against a C implementation of the same emulator. It eventually beat Brock, but was very button-inefficient and would've taken 10 hours to play back under Nix, if it would've even made it that far. On my implementation, if I turn the GC off, the much shorter script in [3] OOMs on a box with 128gb about 2/3 of the way through!
What a delightfully insane abuse of Nix. I enjoyed reading that immensely.
spwa4 2 hours ago [-]
Why are people so enamored with lazy languages? There's an obvious and easy way to build lazy evaluations in any imperative language: return a data structure with a .run() method instead of the result.
It's the basic principle behind the ML libs for instance, none of which are made in a lazy language since it buys you nothing but really adds complexity (you can no longer do the trivial reasoning "if I ask for A B C, then A happens, then B, then C", the order is never guaranteed and these are the errors you constantly make in languages like haskell and prolog, or at least that goes for me).
This also means you can no longer explicitly optimize. And if compilers were infinitely more intelligent than me that would be fine ... but of course when it matters I can beat every compiler I've ever seen.
__MatrixMan__ 9 hours ago [-]
Wow, what fun!
On a more boring note... I could imagine using this pattern for documentation screenshots. It would be a neat sanity check during PR reviews to show whichever screenshots were changed by that PR in context with their surrounding docs.
Maybe they're meaningful, intended, changes which could be useful context for a reviewer. Maybe they're broken, indicating not only that you now need a different derivation to get the desired screenshot, but also that your prose probably needs to change (since the path to get from start to the screenshot has now also changed).
pdude444 11 hours ago [-]
I think this was a good starting point, excited to see how this grows
vatsachak 12 hours ago [-]
Some of us use nix as a package manager. And others...
MiroslavPokorny 6 days ago [-]
Im confused whats the point of this ?
0xcde4c3db 14 hours ago [-]
The implementation is superficially silly, but I think this can also be read as a meditation on exactly what distinctions we're making when we use terms like "program" vs. "process" or "input" vs. "dependency". Perhaps a cousin to the idea that compression formats represent specialized instruction sets for bitstream generation, or to static single assignment. It makes explicit what is usually implicit in emulator "movies", that each frame has the state of the prior frame as a dependency, even though that relationship is normally thought of as a side effect of the passage of time.
The underlying principle of translating those implicit temporal dependencies into explicit data dependencies very well might have applications in optimizing high-accuracy computer/console emulators. The usual implementations iterate over some variable that represents the passage of time and compute all possible side effects, which is expensive. Many compilers already do a related static optimization by eliminating intermediate variable mutations that can be proven to have no side effects.
(also, if you're not familiar with the "tool-assisted speedrun/superplay" tradition of optimizing video game input sequences as a flavor of metaprogramming, see https://tasvideos.org/NewcomerCorner)
T-R 11 hours ago [-]
To add something a little more practical than "it's fun" or "it's philisophical" -
For someone packaging things with Nix, it's tempting to think of derivations as just a builder - you write your scripts, accept your inputs, and your result ends up in the store.
The article suggests, taking advantage of lazy evaluation, that you can put a whole state machine in Nix (i.e., lift the DSL for describing states into the Nix language). If you had a team of developers that need various customizations of something, they could describe it in Nix, and benefit from the caching of intermediate steps. If someone's just changing something at the end of the long string of customizations, they only need to build their changes.
A more concrete, if very niche, example - Back in the days when we were all training our own deep learning models, my company had assembled a list of facts for every product we could find on the internet. Our data scientists wanted to be able to test datasets with different aggregates of the lists - we needed to both build the training set, and compute it at runtime for new products. There was some compositionality to the aggregates - one run might use the mean, another would subtract the mean from every element, etc. It was time consuming and annoying to manage. One of our developers made a DSL in Haskell to describe the set of aggregates desired for a given dataset, and needed to build caching for each column. The technique the article suggests could have done this out of the box (at build time, even) - build a function and a vector for each aggregate, throw them in the nix store, and then wrap them up as a training set and a server.
kjs3 15 hours ago [-]
It's 'fun', for some folks 'nostalgic', possibly even 'interesting', in a casual sort of way. You apparently aren't familiar with these concepts. No need to hate on it.
esikich 14 hours ago [-]
Not parent, but I am familiar with those concepts however I don't know Nix, so I don't know what I'm looking at.
phoghed 3 hours ago [-]
Did you ever find out or did you give up
kjs3 14 hours ago [-]
[flagged]
esikich 13 hours ago [-]
Idk man, I had the same question. No need to get angry.
kjs3 13 hours ago [-]
[flagged]
esikich 12 hours ago [-]
The irony here is thick.
crikeyhn 12 hours ago [-]
[dead]
ludr 16 hours ago [-]
It's just a fun hack. I think it's neat.
semiquaver 15 hours ago [-]
Answer honestly: what is the point of yourself or anyone for that matter?
I gave 3 talks that I will update on my site[2] once they released.
[1]: https://nix.vegas [2]: https://fzakaria.com/talks
Pokemon was so slow that coming up with TAS scripts, even with state saves/loads for every run, became infeasible, so I ended up searching against a C implementation of the same emulator. It eventually beat Brock, but was very button-inefficient and would've taken 10 hours to play back under Nix, if it would've even made it that far. On my implementation, if I turn the GC off, the much shorter script in [3] OOMs on a box with 128gb about 2/3 of the way through!
[0]: my claudes [1]: https://github.com/psyclyx/nixboy [2]: https://x.com/psyclyx/status/2066249660046725213 [3]: https://github.com/psyclyx/fix#a-less-rigorous-benchmark
It's the basic principle behind the ML libs for instance, none of which are made in a lazy language since it buys you nothing but really adds complexity (you can no longer do the trivial reasoning "if I ask for A B C, then A happens, then B, then C", the order is never guaranteed and these are the errors you constantly make in languages like haskell and prolog, or at least that goes for me).
This also means you can no longer explicitly optimize. And if compilers were infinitely more intelligent than me that would be fine ... but of course when it matters I can beat every compiler I've ever seen.
On a more boring note... I could imagine using this pattern for documentation screenshots. It would be a neat sanity check during PR reviews to show whichever screenshots were changed by that PR in context with their surrounding docs.
Maybe they're meaningful, intended, changes which could be useful context for a reviewer. Maybe they're broken, indicating not only that you now need a different derivation to get the desired screenshot, but also that your prose probably needs to change (since the path to get from start to the screenshot has now also changed).
The underlying principle of translating those implicit temporal dependencies into explicit data dependencies very well might have applications in optimizing high-accuracy computer/console emulators. The usual implementations iterate over some variable that represents the passage of time and compute all possible side effects, which is expensive. Many compilers already do a related static optimization by eliminating intermediate variable mutations that can be proven to have no side effects.
(also, if you're not familiar with the "tool-assisted speedrun/superplay" tradition of optimizing video game input sequences as a flavor of metaprogramming, see https://tasvideos.org/NewcomerCorner)
For someone packaging things with Nix, it's tempting to think of derivations as just a builder - you write your scripts, accept your inputs, and your result ends up in the store.
The article suggests, taking advantage of lazy evaluation, that you can put a whole state machine in Nix (i.e., lift the DSL for describing states into the Nix language). If you had a team of developers that need various customizations of something, they could describe it in Nix, and benefit from the caching of intermediate steps. If someone's just changing something at the end of the long string of customizations, they only need to build their changes.
A more concrete, if very niche, example - Back in the days when we were all training our own deep learning models, my company had assembled a list of facts for every product we could find on the internet. Our data scientists wanted to be able to test datasets with different aggregates of the lists - we needed to both build the training set, and compute it at runtime for new products. There was some compositionality to the aggregates - one run might use the mean, another would subtract the mean from every element, etc. It was time consuming and annoying to manage. One of our developers made a DSL in Haskell to describe the set of aggregates desired for a given dataset, and needed to build caching for each column. The technique the article suggests could have done this out of the box (at build time, even) - build a function and a vector for each aggregate, throw them in the nix store, and then wrap them up as a training set and a server.