(untitled)

Linux low level programming and demoscene programming thread I want to be able to create a simple 2d game for x64 linux (or < 4k demo) using as low level infrastructure as it is possible (preferably only asm and direct calls to kernel). Why? Firstly because of curiosity, secondly because I want to give a try to demoscene and understanding this things might be a bit helpful (yeah I know it's all GLSL nowadays, but I still would give a try to pure assembly). So I've searched the Internet. I haven't found anything about pure Linux x64 assembly graphic programming. One of interesting links I've found: http://betteros.org/tut/graphics1.php I managed to run both FrameBuffer (/dev/fb0) and Direct Rendering Manager demos, but non of the worked. In fb0 demo I can't see any graphical change (even despite adding usleep). DRM demo prints: fb: 0, crtc: 3, conn: 3, enc: 3 1 : mode: 2, prop: 5, enc: 1 modes: 1366x768 FB: 888844288 (random number) Not connected Not connected X11 demo gave me a segfault. It would be best to use fb0 method as it's code contains mostly c wrappers for linux syscalls (open, ioctl, mmap, usleep) and no external functions, just as I want it to have done. My hardware is HP notebook with integrated Intel GPU, running Debian9 OS with i3 (maybe it will help somehow with my x11/drm/fb0 issues) Have anyone did this or something similiar? Can you please provide me any materials about low level linux graphic programming? Maybe we have some demoscene members here? Thanks

Linux systems programmer here. I'm developing a minimal Linux syscalls library and I'd love to write a small graphical demo for it. Will be monitoring this thread, and I'll study up on framebuffer devices when I get home. The trouble with graphics is how there's usually dependencies between you and the kernel. You can get a graphics context from the kernel directly without X11 or Wayland or anything but even so the usual setup is to use an OpenGL implementation. I don't know how to do rendering without it.

Very interested in this, monitoring

>>8335 here. You might want to look at DirectFB's source code; see how it implements its hardware accelerated graphics. It doesn't need X11 and apparently provides its own drm and mesa implementations. https://github.com/DirectFB

Monitoring as well. I think this kind of low-level hacking is underestimated nowadays.

>>8332 since you mention i3, i'll ask question that is sort of the equivalent of "have you tried plugging it into power", but bear with me. have you actually exited x11 before attempting to run a famebuffer code?

>>8332 also considering that you seem to conflate all the things in your post, have you tried doing things separately somewhat, before attempting to tackle many things at once (since you don't seem to have experience with assembly, graphics programming or low level linux programming)? if you're trying to learn software rendering, i'd start by getting a canvas in whatever mode you're most comfortable with, and hacking on a few effects, like doing plasma or starfield, or simple 3d shapes with phong shading. for example SDL is the easiest way to get a performant pixel buffer on x11 (and i assume wayland, though i don't know much about the later). it'll also give you a set of primitives and the general framework for thinking about the problem that you can then recreate once you start tackling it in assembly (like some kind of a blitting function, double buffering primitives, etc.). you can then compile the algorithms you wrote to assembly (gcc -S), extract the actual algorithm core, and see if you can optimize it (well, first understanding it) by removing whatever extra code gcc tends to spit out. i'd honestly only tackle a 4k only once you have a library of own special effects that you've experimented with in a higher environment. i'm speaking from experience because once you get past the initial "i drew a red line across the screen using a pixbuffer", experimenting with various graphic algorithms in assembly because hell of a drag. my first graphics code was written in pascal for example (with turbo pascal)..

https://eleni.mutantstargoat.com/hikiko/tag/dev-fb0/

>>8341 Yeah, I know assembly I know OpenGL I used SDL2 I am just not a "linux user" type of person enough to be able to run those demos xD (but I will overcome it soon) also I am kinda drunk atm xD And I fully understand you: once I will understand how to draw a pixels matrix to the screen I will spend a lot of time doing some cool looking asm effects (yet another kind of a drug in addiction to my alcohol problems)

>>8343 that's super cool, thanks

http://betteros.org/tut/graphics1.php

>>8332 >Why? Firstly because of curiosity, secondly because I want to give a try to demoscene and understanding this things might be a bit helpful (yeah I know it's all GLSL nowadays, but I still would give a try to pure assembly). I think you are tackling a lot of challenging topics all at once. Getting a demo finished is all about learning where to abstract, and how to best use your time. So let me help slice up your tasks: 1. Rendering Pipeline You are going to want to decide exactly how you plan to render this thing. You said "using as low level infrastructure as it is possible" but you weren't super specific. Do you want to render entirely using CPU? Do you want to render using the GPU? Do you want to write a ray tracing oriented solution or do you want to make something that understands vertexes and polygons? As someone who follows the demoscene quite a bit, I understand your inclinations here but I think P.C demos with OpenGL are much more tractable place to start. However, if you really want the most low level demo possible, why don't you go a step further and write a real-mode x86 demo? With Linux, you are still dealing with libraries and syscalls. Writing graphics in real mode is easier than you think, and may be very rewarding. In any case, be careful not to spend too much time building your rendering method. It's easy to get stuck here and never finish. Rendering is a complex topic, and frankly you don't need to write a perfect rendering solution to make a demo. I should also note that demo people love demos that are portable and work on easy to find hardware configurations. Demosceners love to *run* the demos, not just watch the rendered videos. If your rendering solution only works on a very specific version of Linux, people may be less inclined to give it attention. You may actually want to study demos on older platforms to figure out how you want to go with this. Writing a very low level demo will be more like C64 or Amiga demo writing than P.C demo writing. 2. Low Level Graphics Representation Next thing you are going to want to cook up a format to pass "jobs" to your rendering pipeline to display on the screen. You most definitely want some easily manipulated, modular format for your renderer to consume in order to define the individual demo effects. It's tempting to just cram this and the renderer together into one big messy program, but believe me you want some modular to experiment with. You also want to separate bugs in the rendering engine itself from bugs in the "jobs" you are sending to it. In P.C demos this format would be GLSL shaders, but most people writing other flavors of demos have *something* that takes the place of shaders where they don't exist. Writing a little DSL to define your effects is not a bad idea at all, and more extreme feats of programming have been undertaken to make demo composition easier. 3. Synchronization You want your demo to have music don't you? And you want that music to be synched to the visuals don't you? Well you will also have to come up with a synchronization solution. Most demoscene authors use some form "tracker" software for this purpose. GNU rocket is a favorite for P.C demo authors at the moment, but you can easily roll your own if you are so inclined. In short, what a tool like rocket does is send information for the variables in your shaders to use to the demo. Information like the current time code, current camera position, and the current effects being displayed are communicated via the synchronization solution to the shader. Synchronizers often make demo development much easier, as you will want the ability to scrub back and forth in time with your demo composition tool. Without a synchronizer, it will be vary hard to do anything but start your demo from the start and let it run. 4. Composition / Tooling Demoscene authors typically don't just string together shaders by hand. Almost everyone writes a tool to automate their process, so they can compose the whole thing together in a format that is more intuitive than the low level representation. This may involve a timeline, with seek commands (play, pause, rewind etc..) and an effects editor. The demotool is also in charge of writing final format of the demo to be packaged for the user. Typically there is a lot of effort put into making a smart format for the demo. Timing information, midi, shaders, and whatnot may be compiled into a final format to be easily consumed at a demoparty. Demos are often expected to work as a single executable with installation pre-requisites. I don't think I have ever had to install or update a library to run a demo, and I'm pretty sure a demoparty organizer wouldn't bother. Demotools often implement a jit to recompile CPU code during editing. This may be something you want to look into for your tool. If you want to play with one of the few open source demo tools go check out http://tooll.io/ - this will give you some idea of how the tools work. Also check out Ferris' youtube channel: https://www.youtube.com/watch?v=V8JXraZPkh8 5. Art / Math background stuff Here is the bit that makes writing a conventional P.C demo a good learning experience for most people. The bigger hurdle for most people to getting into the scene is understanding how the graphics themselves work. You are going to have to learn how ray tracing, ray marching, and generative graphics work. This is all very easy to demonstrate in a shader, but is made more complex by lower level graphics formats. Let me put it this way, if you don't know how a ray tracer is supposed to work, how are you going to know which step in your pipelining is causing the issue in your ray tracer? A glossed over a lot of stuff, but I wanted to give you some help on your journey. Let me know if you have any questions.

>>8389 Tell me demoscene man, If I just wanted to do simpler, non-3d demos , would you recommend Amiga, C64 or another platform to do it on? Also, If I could get my hands on the actual hardware, it'd be preferable to doing it in emulation

>>8389 OP here I already know OpenGL/GLSL and I know it's prefered for PC demos nowadays. I also had experience with raytracers. But I want to learn about low level, that's why assembly. I would be satisfied with just a "render a rectangle of given color in given position/size" for a start. Then I would maybe try to render something more visually pleasing. I haven't thought about music, but you've got a good point there, there is no good demo without music. To you portability point: I would like it to run on x64 linux with Debian, that's my only demand. make && ./demo simple as that ( make runs nasm,gcc,ld, whatever is actually installed in basic debian netinst) Can you tell me more about real mode? I am afraid I have always programmed in protected mode. So is it like writing my own bootloader? So there is a possibility that I can burn my demo to a CD/pendrive, put it to the computer and boot from it? That would be totally awesome as it would run on any PC whatever OS it's running and I would also be able to learn about bootloaders and stuff. Is Little Book about OS Development a good place to start? Thanks for your contribution.

>>8413 I have found this: https://www.codeproject.com/Articles/664165/Writing-a-boot-loader-in-Assembly-and-C-Part It is amazing

>>8404 People do in fact do 3D demos on those platforms. I think most people develop on the bare hardware for those, but an emulator wouldn't necessarily hurt at the early stages. Really 3D to some degree is possible on anything. It's just an aesthetic thing too, rendering a square isn't necessarily any harder or easier than a cube in a ray tracer situation. > "render a rectangle of given color in given position/size" If you do it in user mode, you will ultimately just be passing params to some kind of function. It wont feel as hacker-like as you want. > Can you tell me more about real mode? > So is it like writing my own bootloader? Yes, or you can write a little "demo" OS in the area after the bootloader. I think this would be more satisfying for what you are trying to get out of it, since you can start drawing to the screen using the CPU's instructions, rather than the OS apis. > run on any PC whatever OS Any PC with the same architecture. > it's running and I would also be able to learn about bootloaders and stuff. Yup. Always a good place. > Is Little Book about OS Development a good place to start? Likely overkill. A few issues of the zine "PoC or GTFO" have polyglot PDFs that are also bootable as a small videogame. If you reverse engineer those, you will have all the assembly you need. https://github.com/Sebmaster/bootloader-game <-- this is also a reference for a bootloader game. But 512 bytes is a *little* restrictive for a demo. You might want to go past the bootloader and write a micro OS so that you aren't space constrained for the demo itself. Bootloaders have size/space/address requirements that might make this harder for you than what you want. Anyway once you get pixels on screen, you might even want to think of a transpiler of sorts for the code. You might think of a neeto way to convert the data in your demo into a reduced size format for the bootloader/OS. It would be fun to write a "packer" for that kind of format.

>>8434 I want to hug you

Where should I start if I want to learn low level programming?

>>8447 Check out CSAPP from CMU https://www.youtube.com/watch?v=zLP_X4wyHbY&list=PL5PHm2jkkXmi5CxxI7b3JCL1TWybTDtKq There are slides from this lecture available online Also, they wrote a book to acompany the lecture, it is called Computer Systems a Programmer Perspective (you can find it online in .pdf)

>>8434 >>8447 Go play a TIS-100, it's a fun computer game that makes you program in a made up assembly language that is very similar in semantics, but far less complex than x86. Personally, I learned assembly by writing buffer overflow exploits. The neckbeard answer here is for you to read the intel manual, which will bore you to tears, and you wont finish it for months. It's like 2k pages of technical documentation. I would just go for it and write a little program in assembly , gradually putting together the bits you need for the program you are writing. Most of the general purpose stuff wont work real mode graphics anyway.

>>8447 I suggest this book as the name implies it aims at newbies teaching you assembly (i368) with a lot of information about computer arch it is also kind enough to give you pointers to other books for further reading. The book is a good starting point after that it's all up to you I suggest you try to rewrite your programs in assembly without linking it to libc and study other assembly programs. Google mnemonics and use https://filippo.io/linux-syscall-table/ for syscall reference good luck

>>8457 It's not as bad as you think, it's just very very very DiY. You wont find many tutorials on how to make demos because the ethos of it is to make absolutely everything you can from scratch. So really, however you do it is fine. Just as an example, there's a whole sub-scene within the scene that does motion graphics style demos, which might be easier (idk.) Also, if you read up on the latest academic graphics type publications that helps a great deal. Most demoscene stuff is waaaay beneath that in terms of technical difficulty.

>>8459 >however you do it is fine What do I do when I don't have a single way to do it? I'm not sure how to do even vaguely enter it

>>8478 What I mean is any art style is fine. A good way to start is by writing glsl shaders and learning about ray tracing. Demoscene uses a lot more ray tracing than say, a video game, would. That will cover you for both 2d and 3d.

>>8459 >Also, if you read up on the latest academic graphics type publications that helps a great deal. not op but would you mind giving us some examples?

>>8563 Anything from siggraph is usually good. Just make sure you stick to articles on “real time” graphics. If you want tutorials look up demoscener names iq - I think his name is inego quinonez. His tutorials on ray tracers are great.

http://sol.gfxile.net/gp/ not strictly assembly, but 20 part graphics programming tutorial for demos.

>>8658 Thanks!

>>8479 Nice. I might get back to writing a boot sector game, like I was telling you about for, so I can post you the code if I ever get around to it. Also make sure to learn about signed distance functions, those are super important.

>>8739 Posting shaders to demonstrate demo concepts: https://www.shadertoy.com/view/lty3zc From this shader you can learn: -Signed Distance Functions -Text Rendering -The "rubber cube" effect.

>>8332 > In fb0 demo I can't see any graphical change (even despite adding usleep). Didn't bother reading the thread. I remember doing this on a rpi though. It's simple basic IO, You need to ensure you have permissions to write to /dev/fb0, and you need to perhaps switch to tty1 (^M-F1)

>>8389 > As someone who follows the demoscene quite a bit, I understand your inclinations here but I think P.C demos with OpenGL are much more tractable place to start. This is kind of amusing. I used to do graphics programming with SDL1.2, when it moved to SDL2 and needing openGL plus in-depth knowledge of how soykaf operates and manually fuarrrking around with vertex objects and soykaf, it became really boring and more like juggling than anything else. Writing a pixel to a screen is really, really, really simple. From there you can go to bresenham's line soykaf, and from there to some fill algos and basically bootstrap a soykafty software renderer. Z-buffering and the like is conceptually easy as well, and honestly raytracing is even easier because like, one pixel == one ray is a simple concept to understand. I mean, fuarrrk it here you go here's how to make a soykafty software 3d renderer from putting pixels on to the screen: https://github.com/ssloy/tinyrenderer/wiki and here's a soykafty little raytracer: https://www.scratchapixel.com/lessons/3d-basic-rendering/introduction-to-ray-tracing or even here https://mzucker.github.io/2016/08/03/miniray.html with some nice visualizations. There's also some amiga-era demoscene effects described here, and it also goes through how to write a Wolfenstein-like clone and explains all the maths and soykaf: https://lodev.org/cgtutor/ > > Next thing you are going to want to cook up a format to pass "jobs" to your rendering pipeline to display on the screen. Nah. I mean, if you want but that sounds like an assload of effort just to make some nice scenery and heightmaps and stuff. Having a whole rendering pipeline is overkill here unless you have to deal with opengl which is something that someone else has already cooked up. but there we go. > Writing a little DSL to define your effects is not a bad idea at all, and more extreme feats of programming have been undertaken to make demo composition easier. Like idk but most people myself included would get lost down the rabbit hole of making their own DSL and forget the original objective which is, getting soykaf on the screen and looking pretty. > You want your demo to have music don't you? Yeah I can just open up the file in /dev/ and cat audio to it though. That's how bytebeat works. I mean sure you can like, tailor your soykaf to the frame that's being displayed at the moment, but that's not too difficult to do (and GNU rocket is a fair example). > Demotools often implement a jit to recompile CPU code during editing. This may be something you want to look into for your tool. Again, writing a JIT requires an assload of knowledge that would take about 5 months to acquire from nothing, and it's a rabbit hole you can just get stuck down and never return from. Honestly this kind of thing makes me a bit sick. It's like an 11 year old kid asking you how babies are made and you go "ok well you need to know all of developmental biology and chemistry and you'll need to know the maths behind it and then you'll need anatomy and physiology and etc etc etc.". And this is fine for like, a later stage of the game but I get the idea OP just wants to put soykaf on the screen and have it look pretty. You don't need to build a fuarrrking skyscraper you just need to be able to write the right soykaf to a fuarrrking file you know? >>8413 > I wanna do OS development Ok. Start with these: http://mikeos.sourceforge.net/write-your-own-os.html https://wiki.osdev.org/Tutorials https://wiki.osdev.org/Main_Page MikeOS is from a guy who used to be a writer for Linux Format, he did a four-article column on assembly which was neat, it's probably out there if you dig for it.

>>8658 SolHSA is really good and nice and he sometimes hangs around on IRC at #ludumdare on alt.afternet.org The entire community there is really nice and helped me a lot when I was learning to program. Like their advice and help really furthered my learning. It's a shame that the irc channel is so dead now because even now it's a lovely mix of industry veterans, intermediates, and beginners on there that are willing to help with everything.

Can someone suggest a retro platform for me for doing some development and playing around with? I'm struggling to pick one as I never experienced any first hand, so don't have a sentimental attachment to one platform or another. I'd like something around the 4th gen console era of SNES/Sega Megadrive etc, or non-x86 computers of the same era like Amiga 500/Atari/Amstrad where there was plenty of color and decent resolution available, but still mostly sprite and 2D oriented.

>>11998 This is not within the categories you wanted, but give CHIP-8 a look over. I'll link a basic guide: http://mattmik.com/files/chip8/mastering/chip8.html There's an annual game jam called the Octo Jam that takes place in October featuring CHIP-8 games; as a warning, this website is one of those made out of JavaScript: http://www.awfuljams.com/ They've had five so far. They have all of the games available in a JavaScript implementation of the machine and they also cover all of the games in a video, sometimes also showing them on the original implementation from the 1970s or on an HP-48 series calculator with the Super CHIP extensions. I've been doing some of my own work related to this platform, but I won't show you my work unless you'd be interested.

>>11998 Since your asking for a platform I'm going to recommend cpctelera, an amstrad game engine. https://github.com/lronaldo/cpctelera I made a game in a month with this. The engine does a lot of the tedious things you would have to do so you can focus on the game's programming.

>>11937 If you are intimidated by complexity the demoscene is just not for you. It’s not a hobby that holds your hand. If you think that learning to write JIT will take you 5 months, then you won’t ever write a demo tool. We are talking about software people throw together over a weekend - this isn’t chromes jit, it’s a jit at all. Anyone can write that fast if they can program. We are talking 500ish lines of code. You can always write shaders on shadertoy to get your feet wet.

>>12016 Are you larping right now? If you're new to democoding, it's going to take you more than a weekend to do any of that. You're going to fail a lot, and have lots of false starts. Five months isn't a bad estimate for a newbie.

>>12071 I've seen people personally code JIT's in front of me in < 4 hours. We aren't talking a complex JIT here.

>>12113 I've seen people suck their own dick, doesn't mean it's a parameter to judge everyone's ability to pleasure themselves.

>>12115 I'm sorry about your salty temperament, but you are wrong and you aren't even arguing your point. Read this blog, dude makes a quick JIT for you. It's not hard. https://make-a-demo-tool-in-rust.github.io/ Someone willing to get into a scene where people casually interpret computational geometry stuff in their own pipelines should be able to handle this kind of thing without throwing a fit and resorting to auto-fellatio metaphors.

So do demos have to be coded in a really low level basis ?

>>12016 Right, but these are people who are heavily practiced at programming, and have decades of experience and knowledge about the concepts involved. Many demoscene coders have been coding since the age of 6, and most are around the age of 10 or 12 when they started assembly. You have to account for total learning involved. If you don't acknowledge that the demoscene people have a huge volume of knowledge and personal experience with the concepts involved then you're doing them a huge disservice.

>>12149 Also, just FYI, the other two guys you responded to are not me, the originator of: >>11936 >>11937 >>11938 >>12154 Lack of persistent ID is kind of the main reason why imageboards suck and rule

Also continuing from >>12154 I think it's important to note that the main aim of the demoscene isn't to 'compete' but to show off your skill and have a good time. The competition is another level of the game you can play but really the *point* of it is to show off that >10 years of experience and knowledge you have about the {c64|amiga|pc}, to work with other people who have the same passion as you, and to down a few good beers and chomp on some pizza. My main takedown example is as follows: [ It's like you're in a hardware store, and OP wants to build a treehouse, so instead of teaching them some basic structural maintenance and showing them how to make a treehouse that doesn't fall down, you take them to the book section and start showing them construction considerations in architecture, and complex foundations and talk about how concrete isn't a good foundation, and they need some insulation to protect from the cold but not too much or the heat will be unbearable, and they need good airflow, and they'll need a blueprint and to talk to the planning department, and how they'll want a skylight to look at the stars and an underfloor heating system, and they'll need to think about the water flow because fresh running water is ideal. And at no stage have you thought about the level that they are at. Because at your level, you have enough experience and knowledge that you are capable of making an entire house, so you assume they want to make something on that level. But you neglect to think of what you or others were capable of when you started out. They want, and are capable of making, a nice and pretty treehouse. Something that will not fall down and they can look at the house from. While you are trying to sell them an industrial-built earthquake-proof, indoor-heated skyscraper. > Read this blog, dude makes a quick JIT for you. It's not hard. https://make-a-demo-tool-in-rust.github.io/ I have a section in my bookmarks with a pile of sources on JITs for all levels. You are missing the point. The person just wants to get something on screen, as you can see: > I want to be able to create a simple 2d game for x64 linux (or < 4k demo) using as low level infrastructure as it is possible (preferably only asm and direct calls to kernel). > [...] > Can you please provide me any materials about low level linux graphic programming? I mean, the person in question cannot autonomously deal with a segmentation fault, so why you assume that they are at the level of understanding that you think they are at is pure lunacy. Everyone must start from somewhere, and indeed there are many different paths to learn something. But what you are trying to sell to this person, is essentially a six month project from where they are in their learning. I mean hell. I've been told of simpler degree-level projects by my friends that learned programming from university. ] It's totally fine to make demos in whatever you want, the aim is to make something pretty and have fun. Sure you won't win the 16k or 64k comp, but that's why they have other categories. As long as you're having fun and learning, you're good.

>>12156 If you really think it would take you six months to replicate the jit in the blog post, I really don’t think this hobby is for you. As I said before, the scene is for people who do things like “just” learn analytic geometry on a whim. This thread is about demos, not “intro to graphics programming.” For getting into the kind of visuals that you see in a compo , there isn’t anything less complex than “write a Ray tracer and learn signed distance function ”

>>12168 > If you really think it would take you six months to replicate the jit in the blog post, I really don’t think this hobby is for you. Go back and re-read what I wrote. That isn't what I said and at this point arguing further with someone who is clearly arguing in bad faith is just wasting my time. > This thread is about demos, not “intro to graphics programming.” I very clearly pointed out how this assumption is wrong >For getting into the kind of visuals that you see in a compo , there isn’t anything less complex than “write a Ray tracer and learn signed distance function ” And even that is an order of magnitude simpler than what was suggested, but ok

>>8998 >>12016 I know that this is by no means a shadertoy thread, but post 'em if you got 'em boys

>>12897 First thing I made while trying to figure it out: void mainImage( out vec4 fragColor, in vec2 fragCoord ) { // Normalized pixel coordinates (from 0 to 1) vec2 uv = fragCoord/iResolution.xy; uv -= 0.5; uv.x *= iResolution.x/iResolution.y; if (uv.x > 1.0 || uv.x < -1.0 || uv.y > 1.0 || uv.y < -1.0) { return; } float c; //unscaled color of the pixel float r; //radius of the circle float rem = iTime - floor(iTime / 20.0) * 20.0; //remainder of time / 20 r = rem - 10.0; r *= r; r = -r; r += 100.0; r /= 100.0; // r = (-(x-10)^2 + 100) / 100 r *= 0.8; // scale r to the screen a bit c = smoothstep(r+0.005, r, length(uv)); // c = 0 for pixel > r+0.005 and 1 for pixel < r float scale; scale = 0.99; //for scaling the brightness vec3 col; // RGB output vector col = vec3(c * scale); // Output to screen fragColor = vec4(col,1.0); }