(untitled)

Can we discuss about automated testing? I've being writing tests for one year now, and it has really changed the way I coded. Though I still have to learn a lot, but I'd like to hear from you. What language you are used to testing? What frameworks/tools do you use? What do you have more difficulty testing?Which good practices you have while writing your tests? Is there any kind of test that you don't usually do? Why? I'll start... I generally work with ruby, most of the time I use rspec, capybara, poltergeist, and selenium and vcr. My bigger struggle while testing is when I need to test some JS, it's just annoying. I like to keep the four phase test, dry as most as possible without losing readability, and always have and success and failure scenario. I don't usually do mock, spy and stub tests, sometimes I'm afraid of it turns into a false positive. Actually I only do this kind of test when I must ensure the same response from other application, what is great for APIs, since you don't need to pay for every test, or even spare you network resources. And how about you, how do you test your code?

I'm trying to write automated acceptance testing for undocumented legacy code that I don't understand. I want to blow my brains out.

I am just getting into unit testing C and i have been playing with GLib's testing framework. Does anyone else unit test C ? What framework do you use ?

I don't use unit testing at all. What do you think of the opinion that unit testing is primarily for those languages lacking strong and static typing? I've noticed a much greater change using such a language, but in the more dynamic languages I still never write unit tests; I'd rather test the program manually after reviewing the code.

>>15752 Static typing is NOT a reason not to unit test. I find that the biggest benefits of writing unit tests are >sanity checks: Did I forget something obvious?Am I retarded? >writing tests to guide my thinking: Going through the process of writing unit tests really gets my head in the right place ahead of doing the actual implementation. Are there edge cases? What calls should be happening? What might the structure of my implementation be? Is there perhaps a way to design it so that testing is easier? >fixing bugs: Writing tests to prove that a bug exists speeds up my debugging thought process. Insanely helpful for when I don't have a clue what's going wrong, but I know where it's happening. >debugging code that I didn't write: As soon as your project involves more than one person, debugging is no longer the same problem. Suddenly, you have to figure out where things are fuarrrking up in a codebase full of soykaf you didn't even write. You know what would be handy? A list of tests that show you exactly what parts of the code you didn't write are fuarrrking up so that you don't have to stare into the spaghetti void, mouth agape, hoping that a new world war begins and you get drafted so you don't have to feel bad for refusing to fix the issues in your project. None of these have anything to do with silly typing errors. That said, I'm a lazy fuarrrk and I usually hate unit testing.[spoiler][/spoiler]

>>15753 Those are all good points. I had Ada in mind and the latest standard, Ada 2012, has pre-conditions, post-conditions, and type predicates, all of which I failed to mention in my first post. I don't see much of a reason to have separate unit tests when the language itself allows me to test invariants before and after a subprogram is called and whenever variables change in certain ways.

I've been writings some tests for the projects i'm taking part in. The way testing suits me well is like so: - have mock functions, that return mock data, for anything that is not part of the program (DB, web APIs, ...). - directly test the main functions of my program. - some very limited unit testing, for critical parts of the program (check that core algorithm is valid e.g.) This workflows allow me to assess very quickly that the whole program is well formed, and returns valid results, and I feel like this is quite enough. Unit testing might be usefull in big projects with many people, but I don't think I miss much from avoiding it.

>>14411 I test in Python or Racket using the standard library unit test support. Sometimes I'll use Python asserts for a quick script. It's easiest to test functions that aren't dependent on any external state; no IO, no globals, no member variables. When I needed to test IO, I've used virtual machines, containers, and web browser drivers, but I avoid testing IO since it's such a time sink. Like you, I'll also test APIs when I'm writing web services. I've also tested in C. I wrote my own assert function and a Makefile rule for running tests. I was writing code for a micro-controller with a terrible debugger, so the only way to test the code was on my PC. I wrote as much of the code to be device independent as I could, very little IO with the micro-controller error message function stubbed out. Testing has made debugging much easier. When I think a piece of code is causing the bug, I add a test to check the behavior is as it should be, then I run the test. I repeat this until the bug is found. Once a test fails, I fix the bug.

Obviously just don't make mistakes, get good

>>15821 Hi there lainon! Since I read you I am considering doing some racket web testing. Other than the awesome racket documentation, do you have any material on the issue? Maybe an open source project I can draw inspiration? A collegue of mine worte a webpage. The way its written, its impossible to unit test and now I am suffering the consequences. I thought a fun payback project would be to abuse the page

My personal projects are mostly webextensions, which are awful to test because they're distributed: you have different processes interacting with each other and maintaining their own state. So I just check that a few user scenarios work consistently with jest and selenium. Another pain point is that Selenium isn't really intended for testing webextensions, so you can't interact with the browser's UI. And also you can't load webextensions in headless chrome. Absolutely dreadful. I also played a little bit with Neovim's testsuite which is a real pleasure to work with. My day job is to work on a compiler and we use a custom python framework to do that. Compilers are really nice to test because they can be seen as pure functions: take an input A, get an input B, if you get anything else you might have broken something and you need to investigate. Really simple and nice. >>15752 >I'd rather test the program manually after reviewing the code This doesn't scale, unfortunately. >>15844 You can test webpages with selenium.

Am I missing something - is testing anything more than making sure a function works before doing the next one? How can someone be a programmer without doing this? Why do you need frameworks?

>>15871 The testing they are talking about is more of an automated continuous kind. Trying to write test cases to cover as much of your code as possible, and running those tests on each commit / build. This way you can detect regressions when tests fail. Sometimes you modify some part of your code that causes another part to fail without you noticing it. Tests help with that. Sometimes tests are even used as a way to spec requirements. You write a comprehensive test case that covers all your use-cases and failure modes, and then you write code until you make those tests pass. For example, there's an unofficial test suite for the Common Lisp programming language standard that tests compliance for almost all behavior. If you wanted to write a new Common Lisp implementation you can use those tests to help you stay conformant to the standard. The more tests you pass, the closer to the standard you are. Test frameworks help you write and run those tests. Writing tests is not always straight forward, especially when your code depends on specific state or outside I/O (like other web services and such). Some frameworks help you do stuff like creating mock data and such. Tests are sometimes categorized into: * Tests that test an isolated function are called Unit Tests. (For example, asserting that the result of a function that calculates distance between two points gives the right answers for a number of test values) * Tests that test the interfaces between multiple services are called Integration Tests. They ensure that the contracts established between the services are still valid. (Those usually involve cross-services API calls and such). * Tests that try to test the main use-cases throughout the whole stack are called End-to-End Tests (For example, an automation framework that clicks buttons and fills forms on your website to try to sign-up for your service. It runs automatically after every change to make sure this critical user path is not broken).

I build my own unit test system that integrates with my logging and bug tracking systems. For the web I have modules that can run in browser or via cmd line (node.js), both versions (in browser = pic related) generate a text blob which can be parsed by my issue tracking system. In my system verbose logging can be disabled, as can individual test results. The ability to quickly see the test results in browser helps with rapid prototyping, i.e., don't have to run a full test at a command line or wonder if errors only appear in server (node.js) or client (web) versions as I can test those. The resulting blobs can be POSTed to the tracker. For native C projects I have "test" build target in my makefile that generate the same sort of blob, and can be parsed by the same issue tracker. My bug / issue tracking isn't very advanced, it's meant for me, a sole developer, so it has no assignment of bugs to certain devs / groups. It uses a single MariaDB (libre MySQL) instance to aggregate data. However, I can issue "developer tokens" which permit other devs to submit bugs and test results to my system. My C projects use Vagrant VM to normalize the dev environment. A single "vagrant up" command installs the build dependencies (if provisioning is needed), configures the VM dev environment, pulls the latest version of the source and builds the project then executes the unit tests, which (if a dev-api token file exists) then posts the results to the issue tracker. The linux vagrant (VM) dev environment normalizes and simplifies build generation. (no "well it compiles for me, no sure why your build breaks") I can generate exact reproducible binaries, a plus for the security conscious, and cross compilation from the vm means I build for both windows and linux in both 32bit and 64bit binaries. Testing can happen outside the VM, but the generated unit test file still needs to be sent. (a .BAT file for windows executes the vagrant build then runs test.EXEs outside the VM then executes a script inside the VM to parse and send platform info / test results to the issue tracker. This setup gives me cross platform decentralized continual integration. To my knowledge this is something that does not really exist in any turnkey product. That is: I have no centralized build server(s), much like git needs no centralized source repository. The devs who I trust can generate builds and send automated test results to the system. This lets me have a greater variety of test platforms to check code against with a very small centralized issue tracking web service footprint. No having to use a "cloud" of continual integration VMs, like Docker. For larger projects GitLab's CI and docker integration is really swank though.

>>15844 I haven't written any tests in Racket that fit your description. There are some projects on the mailing list you might like. https://github.com/Halfwake/web-sweeper If you want to knock over your friend's site and it's soykaf, it should be enough to trigger a type error. If you're lucky, you'll get dropped into a debug shell after causing a 500 error.