Machine Learning

General thread for ML related topics. Let's talk about our current projects, learning aids, questions, curiosities and forth. Also Forth. I'll start by posting what are generally regarded as the foundational books for the discipline.

I'm going to try and implement an MLP in Lua/Torch, so I can learn the framework. Then I'm very tempted to go nuts and try and rewrite it in Forth. That's gonna be an adventure.

Working on a program to guess the popularity of upcoming anime. Not sure if this will work but its a fun little project. I am scraping data form MyAnimeList and try to build different models to average over. Would be cool to map trailers to popularity via deep learning but I am lacking the computational resources atm. So I start with the classical approach and build some models by hand.

>>324 What do you use as inputs? Because the popularity, for example, often correlates with the staff (ie director) working on the series. You'd probably need quite a large number of inputs to represent that.

>>325 Currently only the scraper for the target values (users, popularity, score) is finished. As for the inputs I plan on using different models. My first ideas are mapping 1. genres and 2. staff to the target. Maybe with logistic regression? But you're right, the data may be of high dimensionality. Maybe I can limit the input dimensions by initial analysis (like removing some unknown voice actors before running the algorithm). I am still quite new to ML but it's nice for playing around and getting a feeling for the tools and algorithms. And it's more interesting than housing prices or tulips.

>>250 Is there a python framework for ml?

>>390 http://scikit-learn.org or https://keras.io for deep learning.

>>390 Most of them, actually. These >>392 then also http://deeplearning.net/software/theano/ , https://www.tensorflow.org/ and Torch has recently released Python wrappers of its code: http://pytorch.org/

There are generated data on the uniform distribution, I need to use the Maximum Likelihood Estimation to find such parameters of the normal distribution, so that it coincides with the data from the uniform distribution as much as possible. The lecture has many mathematical formulas that I can not translate into an algorithm, how can I do this?

>>788 >The lecture has many mathematical formulas that I can not translate into an algorithm It's actually just a couple of loops. When fitting a Gaussian all you need to do is to find its mean and variance. The mean is estimated by the arithmetic average of your samples (x^), the variance is also estimated from the samples as sum((x_i - x^)^2)/n for i=1..n. Compute those two values and you're done. MLE leads to very simple results for normals. This, of course, assuming I've understood your exercise. You've basically got a bunch of uniformly distributed data and want to fit a normal over it, correct?

>>246 starting on my research to develop an ML algorithm to classify lung tissue slides as cancerous or noncancerous this tuesday, undergraduate project

Anyone here work/do research in deep learning? What's it like? Also, currently attempting to solve OpenAI gym environments with reinforcement learning using tensorflow. Admittedly haven't had too much success yet..

>>1318 https://www.kaggle.com/c/data-science-bowl-2017 Is it related to this? Apparently there's fat prizes for scientists who manage to detect lung cancer with CT-scans.

The Deep Learning book by OpenAI seems to be the most universal for the topic. The Peter Norvig book is most people's goto for Artificial intelligence. Is there a good resource that is in depth for machine learning in general? The O'Reilly book on python and machine learning seems more general from the bits I have had the time to read. Can someone advocate for the mitchell book or offer alternatives? Thanks

>>1358 nah it's off of tissue, not CT scans and the entry for that's already passed

How highly valued are these particular programming skills? Is it worth it to, let's say, get a master's degree specializing in neural networks? Or is the fancy education not worth it unless you're going straight into research? Sorry if this question is too meta. Either way, bumping the thread.

>>1518 using NNs just requires you to know the libraries you're using, it's not really programming intensive. education past a BS in general isn't worth it unless you're going into research. that's what graduate and beyond level education is for, getting published in papers and doing research

>>1361 > The Deep Learning book by OpenAI seems to be the most universal for the topic. I didnt know this existed I did a search for it but what i found is from MIT press. Do you have a link to this? this is what i found http://www.deeplearningbook.org/

>>1519 Jesus Christ, no. There's a colossal amount of theory behind machine learning and you MUST know it to make things work properly, never mind smartly. It also requires at least some familiarity with linear algebra and numerical analysis. If there was a bachelor's specialized enough, I guess it would work alright, but I don't think they exist. Ultimately, it's a specialized field but it's getting pretty ubiquitous. It's not exactly as required as knowing algorithmics for a computer scientist, but imho it's getting there. tl;dr yes, fancy education is worth it.

>>1534 >Jesus Christ, no. There's a colossal amount of theory behind machine learning and you MUST know it to make things work properly, never mind smartly. It also requires at least some familiarity with linear algebra and numerical analysis. Lets be clear about this, the 'theory' you are referring to is ALL MATH. The math you have to know goes well beyond some cursory linear algebra and numerical analysis. There are some machine learning books that are light on math and will give you cookie cutter algorithms to perform machine learning tasks to give programmers with little math background a feel for what is going on, but that kind of knowledge will not land you a job. Most the people who who land jobs in machine learning in Silicon Valley are not CS graduates but people with degrees in other areas of science and engineering who apply the knowledge they have in Matlab, SAS, SPSS, R, etc to the now lucrative field of machine learning.

>>1554 >Matlab, SAS, SPSS, R 75% anti-freedoms. Though unfortunately true for many engineers and then statisticians forced to use SAS because of contracts their company or government has with corporations when they want to just use R. There are lots of physicists that use a lot of python, C/C++ and Fortran there too. There is considerable overlap between the foundations for ML and the mathematics and numerical methods physicists learn in their degrees.

>>1554 >Lets be clear about this, the 'theory' you are referring to is ALL MATH. All the more reason to require specialized training. In fact, that physicists and others get ML jobs is exactly because they have a much more solid grasp of statistics, algebra, calculus and error theory than your average CS guy.

>>1562 Once weve established the fact that machine learning is entirely based on math, that doesnt mean that you cant get an intuitive non-mathematical understanding of how it works. Im surprised no one has uploaded this book so I will, its the best book for learning machine learning (which is a subset of data mining) without having to learn the math. The program used to analyze data is called Weka which is a free multiplatform program that was developed by the authors of the book. Its much easier than having to learn R like in most other machine learning books.

>>1563 >GUI tools I guess we're talking very different jobs and levels of task complexity here. There indubitably is market demand for data miners and possibly for very pedestrian machine learning people (I honestly have many, many reserves on the latter), but the other lain's question was if it was worthwhile to pursue specialized training in the field, which imho it is without doubt.

>>1571 yes of course, if you want a career in machine learning you have to learn the math, you cant fake it with general knowledge

anyone have any good resources on kernel convolutions and convolutional nn's?

>>1528 The link you found is correct, that is the "open" version of the book. https://www.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618

>>1534 I think if you took undergraduate coureses focused on the topic and then continued to self teach from text books after graduation you would not need a graduate degree.

are the convolutions in convolutional neural networks dot products between the filter and the local receptive field, or are they weighted averages? i've heard different sources call them different ones of these. e.g. andrej kaparthy from stanford and wikipedia says they're dot products, but computerphile on youtube and multiple academic papers i'm reading say they're weighted averages. they're not the same and i did the math in python to make sure and they're definitely not the same. which is it?

>>1318 Good luck anon, sounds interesting. Are you going only for optical measurements or for chemical measurements as well? Will you be using tissue dyes? If so, what would they highlight?

Slightly off-topic... b-but... do you have any good quote about AI?

>>1915

>>1907 you get the weighted average by dividing the dot product by the number of cells. you're basically just using the euler method to approximate an integral over the receptive field.

>>1917 >>> v1 = np.arange(25).reshape(5,5) >>> v2 = np.arange(25,50).reshape(5,5) >>> v3 = np.zeros(25).reshape(5,5) >>> for i in range(5): ... for j in range(5): ... v3[i][j] = v1[i][j] * v2[i][j] ... >>> dot_result = v1.dot(v2) >>> dot_result array([[ 400, 410, 420, 430, 440], [1275, 1310, 1345, 1380, 1415], [2150, 2210, 2270, 2330, 2390], [3025, 3110, 3195, 3280, 3365], [3900, 4010, 4120, 4230, 4340]]) >>> v3 array([[ 0., 26., 54., 84., 116.], [ 150., 186., 224., 264., 306.], [ 350., 396., 444., 494., 546.], [ 600., 656., 714., 774., 836.], [ 900., 966., 1034., 1104., 1176.]]) >>>

is it a contradiction of values to learn about ML/AI when it inevitably turns into data mining, while I I hate the invasions of privacy that this becomes used for?

>>2227 [I dont know if this is off topic, moderator feel free to move these comments] No its not a contradiction, data mining has been used since before there were computers. Companies have a right to look over their records and target their sales to certain customers. Search sites and social network sites only know the information you give them. They have a right to hold on to that information and use it to tailor your online experience to things you are interested in, thats the whole point of using search and social network sites, isnt it?

>>2227 Commercial applications of data mining are important, but they're only one of many. ML is used in all sorts of fields, from science to healthcare or domotics. Say, a silly case was a project I got told about by a prof, where they stuck motion sensors on the back of tortoises and then used the date to infer and monitor their activities throughout the day. That same technology was later on put to use to monitor the health and needs of the elderly to provide them with automated healthcare services. In funny ways too: they were able to notice how often the assisted person was having guests over and thus see if they were having a properly fulfilling social life. As always, technology is never to blame and always has all sorts of uses.

deep learning book and a build your first neural net book

>>2227 Nah, there are many humanitarian uses of data mining. Virology, for instance. Not to mention, if you learn about AI and stuff you might be able to make a real artificial mind. How cool would that be? Also, if you learn how that soykaf works you can also learn how to combat it. >>2229 >companies have rights good one.

I hoped somebody here would be into cellular automata, hence I made this thread >>2594. I met some people in the past that I probably could have asked, but I didn't think of it at the time, sadly. I want to know how they can be used for machine learning, automated programming and to prove theorems in math. Discussion of them specifically related to machine learning can also go in here.

>>2709 I've wondered in the past if neural networks could be modeled with a form of cellular automata. Like, each neuron activates its neighbors, propagating signals across the network. Of course, there would need to be more complicated rules to model the axon/dendrites, and I guess you might just end up with the layers of neurons you see in machine learning...

>>2775 they can. well, CNNs can. the 'rule' is just the conv kernel.

>>2775 I wonder if CAs can do genetic or memetic programming. I thought about implementing a fully bio-inspired programming paradigm where the structure of the CA actually mimics the structure of genes, with 4 base functions mimicking DNA/RNA building "protein chains" out of computer code. I'm at a loss when I try to imagine how to actually get this done, though. Is there even a library that can handle that?

>>2775 RNNs pretty much do this already, just the notion of "neighbours" is not restricted to immediate adjacency in a mesh. There's also modelling of actual neurons, as you'll easily imagine. Look up the Izhikevich model and Hebbian learning.

>>2783 Maybe before trying to fully develop this, I should try it using only 2 or 3 functions and tailor it to either do something that someone here suggests, or to solve a common programming test like Fizzbuzz. Of course, Fizzbuzz probably wouldn't produce any worthwhile information, given that conventional coding is more than adequate for it.

>>2783 After looking online, I found some papers saying that yes, you can make a genetic program with a CA. You can also do the reverse and make a CA with a genetic program. http://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1413&context=cc_conf_hic https://arxiv.org/ftp/arxiv/papers/0711/0711.2478.pdf http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1094&context=theses_hons

>>2794 http://joelgrus.com/2016/05/23/fizz-buzz-in-tensorflow/ I hope you've read this. Can anyone suggest some practical exercise based guides for ML practice? I love the method of learning by doing exercises, like coding contests and whatnot. What's the gold standard for ML?

>>2793 Are there any good alternatives to NNs for that kind of thing?

https://arxiv.org/pdf/1702.08835.pdf Thoughts on gcForest as an alternative to RNN?

Im currently getting into machine learning from data science and reading "hands-on machine learning with sci-kit and Tensorflow " from O`Reilly. >what would be good read after this? >Should I maybe repeat some of the mathematical aspects of writing algorithms ? etc

>>5488 If a mod can delete that post its cool for I did fuarrrk up formatting. Sharing my own simple as fuarrrk neural network implementation. I'm a beginner so I think it should help other newcomers to this field. Also waiting for your advices ! I think there is no memory leak at all but I didn't verify. You can easily change hyperparameters by altering the arguments of the init_layer function. Actually it learn to perform XOR. To compile and run: gcc -lm -o neuron neuron.c && ./neuron c #include <stdio.h> #include <stdlib.h> #include <time.h> #include <math.h> struct Node { double *weights; double net_input; double output; }; struct Layer { int start; // 1 if this is the input layer. int n; // number of nodes in the layer struct Node *nodes; struct Layer *child; int b; // number of biases struct Node *biases; }; double e = 2.71828; void init_layer(struct Layer *l, int n, int b) { int i; l->n = n; l->b = b; l->start = 0; l->child = NULL; l->nodes = malloc(l->n * sizeof(struct Node)); l->biases = malloc(l->b * sizeof(struct Node)); for (i = 0; i < l->n; i++) { (l->nodes[i]).net_input = 0; (l->nodes[i]).output = 0; } } void init_weights(struct Layer *l) { if (l->child != NULL) { init_weights(l->child); } else { return; } int i = 0, k = 0; for (i = 0; i < l->n; i++) { (l->nodes[i]).weights = malloc(l->child->n * sizeof(double)); for (k = 0; k < l->child->n; k++) { (l->nodes[i]).weights[k] = (double)rand()/RAND_MAX*2.0-1.0; //printf("Initializing some weights (%f)...\n", (l->nodes[i]).weights[k]); } } for (i = 0; i < l->b; i++) { (l->biases[i]).weights = malloc(l->child->n * sizeof(double)); for (k = 0; k < l->child->n; k++) { (l->biases[i]).weights[k] = (double)rand()/RAND_MAX*2.0-1.0; } } } double f(double x) { return 1/(1 + pow(e, -x)); } double *final_output(struct Layer *l) { int j = 0, k = 0; double sum = 0; double *z; if (l->child == NULL) { // output layer //printf("Getting into the output layer!\n"); z = malloc(l->n * sizeof(double)); for (k = 0; k < l->n; k++) { z[k] = f((l->nodes[k]).net_input); } return z; } else { // input layer //printf("Getting into some layer...\n"); for (j = 0; j < l->child->n; j++) { sum = 0; //printf("Computing the net_input for each nodes (%d) of the next layer...\n", j); for (k = 0; k < l->n; k++) { if (l->start != 1) { (l->nodes[k]).output = f((l->nodes[k]).net_input); } sum += (l->nodes[k]).weights[j] * (l->nodes[k]).output; } for (k = 0; k < l->b; k++) { sum += (l->biases[k]).weights[j]; } //printf("Net input of the %d'th of the next layer is %f.\n", j, sum); (l->child->nodes[j]).net_input = sum; } final_output(l->child); } } // n is the number of output nodes double total_error(double *z, double *o, int n) { int k = 0; double te = 0; for (k = 0; k < n; k++) { te += pow(o[k] - z[k], 2); } te = te / n; return te; } void update_weights(struct Layer *root, struct Layer *l, double tres, double *o, int n) { if (l->child != NULL) { update_weights(root, l->child, tres, o, n); } else { return; } double te1, te2, *z; int i = 0, k = 0; for (i = 0; i < l->n; i++) { for (k = 0; k < l->child->n; k++) { (l->nodes[i]).weights[k] += tres; z = final_output(root); te1 = total_error(z, o, n); free(z); (l->nodes[i]).weights[k] -= 2*tres; z = final_output(root); te2 = total_error(z, o, n); free(z); (l->nodes[i]).weights[k] += tres; (l->nodes[i]).weights[k] -= tres * ((te1 - te2)/(2*tres)); //printf("Updating some weights (%f)...\n", (l->nodes[i]).weights[k]); } } for (i = 0; i < l->b; i++) { for (k = 0; k < l->child->n; k++) { (l->biases[i]).weights[k] += tres; te1 = total_error(final_output(root), o, n); (l->biases[i]).weights[k] -= 2*tres; te2 = total_error(final_output(root), o, n); (l->biases[i]).weights[k] += tres; (l->biases[i]).weights[k] -= tres * ((te1 - te2)/(2*tres)); //printf("Updating some weights (%f)...\n", (l->biases[i]).weights[k]); } } } void load_inputs(struct Layer *l, double *inputs) { int k = 0; for (k = 0; k < l->n; k++) { (l->nodes[k]).output = inputs[k]; } } void clean_network(struct Layer *l) { int i = 0, k = 0; if (l->child != NULL) { clean_network(l->child); for (i = 0; i < l->n; i++) { free((l->nodes[i]).weights); } } free(l->nodes); } int main(void) { int i = 0, k = 0, j = 0; struct Layer input_layer, hidden_layer, output_layer; double inputs[][2] = {{0.0, 0.0}, {0.0, 1.0}, {1.0, 0.0}, {1.0, 1.0}}; double outputs[][1] = {{0}, {1}, {1}, {0}}; double *z; double te; srand(time(NULL)); init_layer(&input_layer, 7, 2); init_layer(&hidden_layer, 7, 2); init_layer(&output_layer, 1, 0); input_layer.child = &hidden_layer; hidden_layer.child = &output_layer; input_layer.start = 1; init_weights(&input_layer); do { for (i = 0; i < 4; i++, j++) { load_inputs(&input_layer, inputs[i]); z = final_output(&input_layer); te = total_error(z, outputs[i], output_layer.n); printf("te = %f\r", te); free(z); update_weights(&input_layer, &input_layer, 0.1, outputs[i], output_layer.n); } } while (te >= 0.1); printf("\nCompleted after %d passes.\n", j); clean_network(&input_layer); }

>>5489 >Also waiting for your advices ! You can simply initialize the biases to zero. As for the weight initialization, you can get a bit fancier like http://andyljones.tumblr.com/post/110998971763/an-explanation-of-xavier-initialization

>>5512 Thanks!

>>1916 He said about AI, not about modern western humans. Present Day, Present Time! AHAHAHAHAHA!

Daily reminder that neural nets are magic in the worst possible way and a dead end for ML. Don't be wooed by expensive and well marketed Google vanity projects.

I'm looking for information on radial basis neural networks as it pertains to >>5718 and >>5729. Does anyone here have any experience with them? I've downloaded a ton of papers on them and am currently going through them, so I'd appreciate anything that could speed up the process.

>>5780 Quality post. Might as well discard the standard model for quantum mechanics with that mindset. Even if we don't understand the fundamentals behind deep neural nets, the solutions they provide are still magnitudes superior to previous algorithms and thus (in the real world) better

>>5945 >we don't understand the fundamentals behind deep neural nets what do you mean? it's just eigenparameter extraction. it's an area of active research, sure, but the fundamentals seem well understood.

>>5973 I think he may be referring to the underworkings of a specific neural network. Since the rules a neural network model follows are complicated and extraordinarily difficult, and near impossible, for a human to comprehend due to how the rules are generated, people sometimes look at them as "magic." Essentially a black box. You give a neural network input, it gives you output, and the rules that governed that specific model are never clear. For the opposite case, consider machine generated decision trees: not only do you have the input->output process, you can easily look at the rules the model used and follow along. I disagree with this making neural networks "magic in the worst possible way" as >>5780 is claiming, but it's a common enough claim.

I'd like to learn a new language by small projects, this is one of them: One of my friends is always late from meetings, pub crawls, etc. We came up with the idea to create a program that can give an estimate on his time of arrival. The data we have is the usual people have: - his estimate on his arrival - various excuses for not coming before - if he knows he will be late, by how much - according to him ( a special case of estimate with a condition: already late ) - etc. Basically the estimate should be based on one continous variable and several discrete ones. We will collect data to be able to get more precise estimates. I learnt basic probability theory and I'm willing to learn if needed. What kind of algorithm should I implement? Where to look?

>>366 I think it would be another neat side project to do the predictions based off the studio/staff rather than just the numeric scores. You would have to extremely simplify everything but I still think it would a good project to undertake. You could do that after your current one, refine everything, and then combine them.

>>5973 > eigenparameter extraction Da fuq? No it is not.

Currently learning how to make a twitter "Filter Bubbble" or "User classifier" Will need to learn feature abstraction, Fasttext and Keras/Tensorflow

there are bootcamps and online schools to learn this if you have that kind of money https://www.switchup.org/research/best-data-science-bootcamps https://lambdaschool.com http://www.loonycorn.com/#/content

>>7499 don't waste your money. if you want to waste it, buy me food.

How are your projects doing nowadays? Still alive?

This is probably a stupid question, but I'll try my luck and ask it anyway. I thumbed through the first book in the OP and got lost fairly quickly, what should I study before I try getting into this sort of thing?

>>11028 Siraj Rival has a youtube channel that explains all the math involved in ML in a very easy to understand way, thats a good place to start.

>>11028 statistics and linear algebra

>>11028 There's another book in this thread called "Make your own neural network". I decided to start there. I have read about 1/3 of the first pattern and so far find it very approachable. All we're doing is drawing straight lines and moving them around based on their proximity to certain interesting points on the 2-d plane.

How do i mine data? I want to make this little pet project compare unemployment rate from various coutries with crime rates from all around the world and make some assumptions on it I gathered few books on Data Mining but i still dont know: how do i attain those large datasets? I heard about data warehouses concept, should i look for "free unemployment data warehouse" or something like that? And can i run those algorithms on normal computer or do i need powerful clusters, and for example register some free/cheap servers on AWS?

>>12401 Creatively, considering what you want the data for is how. Depends on data you want. Want stuff on websites? Web Scrapers. Want stuff on social media? APIs/Web scrapers are for that. Want stuff from Government statistics or whatever, they should release those numbers to public for you to find(will vary depending on country) so just converting it will probably be what you want to do since I doubt they all use the same format. How you mine your data will depend on what you want the data for.

>>12401 I used to work doing macro research, and my bachelor's thesis had a similar topic. For that kind of data you dont usually need any big data. Employment rates, if you are lucky, are published in a monthly basis. Same goes for crime rate. Now you have data, if you are VERY lucky, since say 1960. Usually is since 1980 - 1990. Thus, 50 (years) * 12 (months) * 200 (countries) is NOT BIG DATA, and you can do your analysis in your laptop. You cannot fit an ann to this kind of data. To get your data, look into the world bank dataset, us census, eurostats, ceoe, international monetary fund... Those types of organizations tend to have that kind of that. As for methods, look into panel data models.

https://www.youtube.com/watch?v=OQQ-W_63UgQ&list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6 I'm watching these lectures from the Stanford Unviersity in order to make an Neural Network to classify tweets in the political compass. I hope it's useful for someone.

>>12405 >>12408 thanks a lot, maybe you happen to know how to make Web Scrapers in today day and age, when view is seperated from data, and html source does not says anything?

>>12416 Not the same lainon you're responding to but: Most sites that have massive sources of data (twitter, etc.) either have full company-supported APIs, or a reverse engineered API that you can plug in to. Everything else can probably be extracted in the long run by asking firefox to save the html page or maybe using something like import.io, which used to (iirc) have a free tier that allowed you to pull datasets from websites using basically that and firefox's web inspector tool.

I had to work on a segmentation problem. So I fine tuned one dnn I found on github to our data. But it is taking the memory of entire gtx 1080ti. How would you go on making it not use that much resources? is it even possible? or I have to find something else that does not take that much resources in the first place? How do they do it in companies that use dnn in production. What are the standard practices? any resources on this would be helpful.

>>12506 Are you talking about the model using the entire GPU during training, or when you run it? What framework are you using? ie Tensorflow, Pytorch, Caffee, etc. Each framework should have a way to specify the amount of memory used, like Tensorflows per_process_gpu_memory_fraction.

>>250 JFC this is a legit ancient post, I wonder if that anon has jumped ship to pytorch now

Lets say i bought Saphho poems. Most of them are unfinished and i want to use machine to finish them for me. Can i use machine learning techniques for that? Any hints or relevant texts?

Where do you guys receive your daily input of fresh data science related news from?

>>246 Hi Lains, I've just started a project for an automatic ASMR generator. It basically clones the voice of a given ASRMtist from a short mp3 track, and then can produce any given text using a TTS network. I'm using pre-trained models for now, but the results are quite creepy. https://vocaroo.com/i/s0oMuF08sTHn https://vocaroo.com/i/s0LoS9r4qH7A

>>14456 Twitter, Arxiv Sanity, Hackernews, Reddit

>>14459 Hacker news should really be renamed investor news, but it does occasionally have some interesting stuff so I check it. Especially the newest posts page. Seems like that gets more good stuff that hasn't yet been filtered out by their VC demographic who are just obsessed with buzzwords and nonsense and bloodless financial analysis.

I have a massive project where we need to use transfer machine learning, nonlinear point reduction, and unsupervised learning. the classes are it's functional, it's broken, or it's breaking. Some can be clustered but others have a slow degradation. Any advice or similar research?

>>14461 Hell, just drop the name to "news". A lot of crap on there doesn't satiate any hacker's or investor's curiosity.

>>14980 pinch, go2fucking bed

>>12414 Yeah, that's gonna be useful for propagandists and little else. Jesus. I really wonder about some of the clearly aspie programmers who build these things, apparently without ever considering what their use case might become. Like those assholes who decided to try and build ML software that determines people's sexuality from a portrait. Which not only is some kind of weird nineteenth-century pseudoscience, but come on--really? Never thought about how that might get used? By people who obviously won't even care that it might be inaccurate?

>>5772 Like it says, the distinction between those two things grows more slight by the day.

>>15043 >>14980 I'm serious. It's using sensors to detect the degradation or malfunction of specific objects which have different environments and hardware. However, the sensors are the same.

>>15047 a coworker was wanting to do something similar with the same categories. I was joking

>>15053 What did your coworker find?

>>15044 >aspie programmers who build these things, apparently without ever considering what their use case might become. Or they might be perfectly normal citizens trying to earn a living on venture capitalists' money and ignoring their own considerations.

Sup lain, I'm trying to duplicate a cnn from a paper and am having trouble getting it to actually learn. Is there any rookie mistakes in implementation I'm over looking? I'm using tensorflow if that matters

i don't think i have the smarts to understand complex machine learning concepts but i do find it interesting though. an article i find humorous https://techcrunch.com/2013/04/14/nes-robot/ also https://github.com/CorentinJ/Real-Time-Voice-Cloning an idea i had as a project for awhile now is to find a willing participant and get their permission to scrape all the data from all their social media accounts and feed it into a machine learning algorithm to see if an AI can impersonate a human in a believable manner and post my findings and source code publicly online.

>>15044 >not only is some kind of weird nineteenth-century pseudoscience psychologists found out how to determine you hormones (and political affiliation) from your facial features, and there is also some quite good studies showing that hormonal imbalances during prenatal period determines your sexuality All in all, it isn't out of realm of possibility

>>15044 It would be pseudoscience except that it works. I don't have the source saved but the Chinese were able to build a network that tells you if someone is a criminal or not. Others have made a gay detector with like 98% accuracy. It just works

>>15420 >from your facial features that's because nobody in the history of the world would have thought of making their hair like Trump's until now. also because obvious soykaf like buff guy making scorn face wherever he goes is probably a moron affiliated with the republican party. whereas anyone with purple hair is most likely liberal. now if we get past these two obvious cases that we could already solve without your pseudoscience, we'd see that whatever paper you're talking about is mere academic BS and not applicable to any real use case >>15432 yeah i also can tell the last 3 spices you cooked with by the shape of your hair. i cant find the paper, hold on

>>15484 >yeah i also can tell the last 3 spices you cooked with by the shape of your hair. i cant find the paper, hold on New to the thread but cmon lain, these types of responses aren't helping anyone. First result from a startpage search of: "cnn gay detector 98% accuracy" https://www.gsb.stanford.edu/sites/gsb/files/publication-pdf/wang_kosinski.pdf From the first paragraph: "We used deep neural networks to extract features from 35,326 facial images. These features were entered into a logistic regression aimed at classifying sexual orientation. Given a single facial image, a classifier could correctly distinguish between gay and heterosexual men in 81% of cases, and in 71% of cases for women. Human judges achieved much lower accuracy: 61% for men and 54% for women. The accuracy of the algorithm increased to 91% and 83%, respectively, given five facial images per person..." I'm not saying this is enlightened scripture or anything, but the results seem reasonable enough at a glance.

>>15485 Interesting article, but a 91% accuracy is far from reliable. In the current state, this paper can only be used as a "Science proved it" type of moral justification for all kind of purges.

>>15495 It's reliable enough (as in the deviation from 50% isn't just noise) to prove there's a link. Anyone who's not a professional kool-aid drinker already knew this, anyway.

>>15495 If someone isn't going to purge gay people, the study isn't going to suddenly make them do it. The specific model used in the study also isn't available to the general public, so it's not like they can use the network. 91% accuracy is also high enough to justify further exploration in ties to facial structure and sexuality and other characteristics and say that there's a good chance they're somehow linked, so it's far from a useless study. It's not like any study needs 100% positive results to be useful, or even that a negative result is useless, that's not how science works. Your argument is also akin to the slippery slope argument, and is prone to all of the falacies of that argument. Saying we shouldn't do a simple experiment because of the potential consequences is bad science. You don't have any evidence for that. It's like saying no one should travel beyond the mountain because it will make the mountain God angry. In any case the usefulness of the study wasn't in question, the post he was responding to questioned whether the study even existed, which it clearly does.

>>15044 > without ever considering what their use case might become developers and scientist work is not to predict how their tools and research is going to be perverted or twisted by corporations or somebody else, but to develop work that pursue the understanding of reality. Science is neither good or bad, but a philosophical way of understanding the World.

>>15432 >98% accuracy Are that true positives, true negatives or both? I mean, I could publish a paper describing an algorithm that correctly classifies people on heterosexual or homosexual with ">905 accuracy" by just classifying everybody as hetero.

>>15676 *>90%

>>15485 593 Among the top 100 males most likely to be gay, 47 were gay (precision = 47%; recall = 68%). That's a soykaf algorithm however you look at it.

>>15675 This is why I quit comp sci and went into botany. I fuarrrking dare those yuppie pigs to try and oppress people with my eggplants.

Does anyone want to read, in an online study group, Pattern Recognition and Machine Learning from Bishop?

>>2227 you fight ml with ml - algos to detect deepfakes fight deepfakes for example http://www.niessnerlab.org/projects/roessler2019faceforensicspp.html >>15681 farming sector is in deep soykaf. The guys trying to conserve old sorts are under heavy fire from goverments and gmo labs are aiming with modifications to have sorts that grow quick but are way less nutritious as a result. And that flies because of course no labeling for this passes. Consumer can't select on metrics he isn't allowed to see. Situation has been fuarrrked for a while now. Anyone tried to get opengl+opencl ? I've tried it ~6 years ago without much success and it still seems bad now at a quick glance.

>>15684 Wouldn't that just create a virtual war situation? As in how war tends to hasten development, each side has to come up with better more potent weapons than the enemy faster. I.e. just use the algo for detecting fakes as input for the algo for producing fakes. If the fake is detected, negative input, if the fake fools the detection-algo, positive input. Evolve until almost none are detected. Just seems like a way to make the fakes more convincing much faster.

>>15689 you just found GAN's. Generative Adversarial Networks. I think the prevailing problem will be whether the problem of generating believable fakes can be perfectly encoded in a given network size - too small networks thanks to hardware limits will make it impossible to create perfect solutions and thus a prevailing error might continue to get caught.

Does any book covers up to AdaBoost? Or do I have to go to papers and stuff?

>>15710 Forget it. I found one by the authors of the algorithm.

>>15689 That's Hegelian dialectics and it's a fundamental component of the scientific development. In order to create something new, previous contradicting iterations (thesis and antithesis) should be destroyed and merged.

>>13216 bump

If anyone can find or purchase an electronic copy of "The Mathematics of Data" (pic related) please share!

Can anyone help me with a machine learning problem? With boosting (e.g. AdaBoost), why do we use decision stumps? Why wouldn't we just use the weighted majority class of the data points as our classification?

>>14458 that's actually a genius idea, please keep us updated in a separate thread or something

What woul be a good image classifier if you have huge amounts of data and classes, but less time to train?

>>12401 > I want to make this little pet project compare unemployment rate from various coutries with crime rates from all around the world and make some assumptions this has been done quite well by Alternative Hypothesis. They linked crime to IQ, not poverty. Poverty has minimal correlation with crime.

>Poverty has minimal correlation with crime. I would like to see this study.

>>21265 Studies show the opposite is true, the correlation is very strong.

Hey does anyone know what ML magic made these happen? If not ML do you know what process is going on? I was thinking maybe some kind of linear system of equations solving between images and then interpolation.

>>21602 It's using a model called a generative adversarial network (GAN). Specifically, stylegan. When you train stylegan the inputs to the network are random vectors sampled from a normal distribution. Because of this, statistically average random numbers are mapped to statistically average reconstructions of images in the training set (there's more to it than that, but that's the gist of it). You can think of a trained GAN as a continuous(-ish) function mapping the low dimensional input domain (usually called a "latent space") to high dimensional image outputs. So if you take a trajectory/curve in the latent space (usually a spline) and map it through the neural network to the image/output space, you get a continuous morphing like you see here. Here's a thing I wrote a while ago that might be helpful. Idk if it still works but if you look at the code you should be able to get some idea of how this works https://gitlab.com/Vee9ahd1/gantools

>>21605 Big fan of your program! IIRC it only works with the oldest miscellaneous BigGAN(?) category on Artbreeder, and not the newer categories, because we don't have access to the new specialized models(?) Artbreeder is using? And so apparently you can make free animations with GANtools from Artbreeder's misc. category, but not the other categories, which instead you pay a subscription to animate on Artbreeder's website. Have any new models(?) come out which would allow you to make more free animations using your own PC? For example, I don't know if the anime animations in >>21602 were made from Artbreeder images or something else.

>>21609 >Big fan of your program! Huh, small world. Where did you find out about it? I forget if I ever posted about it here. >IIRC it only works with the oldest miscellaneous BigGAN(?) category on Artbreeder, and not the newer categories, because we don't have access to the new specialized models(?) Artbreeder is using? Interpolating StyleGAN works just like interpolating BigGAN (there are some minor differences I can go into if you're curious, but the basic theory is the same). Joel Simon, the guy who made artbreeder (formerly ganbreeder) originally just used the reference model from the original BigGAN paper. This model was trained by google/deepmind and is MIT licensed. My script uses artbreeder's api to download those input vectors I mentioned (latents) and feed them to an instance of BigGAN on your PC. The other models, the ones that use stylegan, were trained from scratch by Joel or other contributors, and were not publicly available last time I checked. I could still get the latents from the api and feed them through a local StyleGAN model, but without the correct weights the output would be different, so there's no real reason to do so. I at one point considered making a "gantools 2.0" that wasn't dependent on artbreeder, but I lost interest before I could come up with a good interface. Maybe I'll revisit that idea at some point. >which instead you pay a subscription to animate on Artbreeder's website Right. I do want to mention that I don't think this is the result of money-grubbing on Joel's part, or an intentional move to undermine my script. The cloud compute bill that site racks up is insane, so he kind of has to charge a fee to cover it. I've talked to him quite a bit, and he's definitely a proponent of liberal licensing and free culture (in fact, I'm the one that convinced him to rewrite the artbreeder EULA so that the renders are CC0 by default). At one point he was open to the idea of releasing his models, and some of them are already published elsewhere (I think the anime faces are literally just gwern's model, here: https://www.gwern.net/Faces), but for whatever reason I don't think he ended up doing it. >Have any new models(?) come out which would allow you to make more free animations using your own PC? Once you write a script to do animations with stylegan, it works for any model. The problem is the interface. Artbreeder gives you a rudimentary way to explore the latent space and pick end points for your interpolation. Without that, you basically have to just pick random points. You can choose the distribution of those points (i.e. how "aberrant" they are, which affects how coherent the final image is), but other than that there isn't a lot of control to be had over the outcome. I can try to dig up the old scripts I used for vid related if you want them, but they're quite messy. >I don't know if the anime animations in >>21602 were made from Artbreeder images or something else. They could be artbreeder or they could just be someone interpolating on gwern's anime model. The former looks like artbreeder for sure, because it would be difficult to get it to be black like that just by randomly picking points. Btw, if you want to talk shop about this stuff, post some way to contact you or send an email to the address I used for my github commits... i think it's still active.

>>21614 >Huh, small world. Where did you find out about it? I forget if I ever posted about it here. There was a thread on here, I think you made it. Shame it 404'd, it had some great animations and info. Not to mention your program deserves more recognition. >The cloud compute bill that site racks up is insane, so he kind of has to charge a fee to cover it. Of course, the website is incredible and I can only imagine that his electrical bill is equally incredible. >I can try to dig up the old scripts I used for vid related if you want them, but they're quite messy. Sure, if it wouldn't be a problem - I'd probably have difficulties using them, but others could find them useful. Maybe I could figure them out at some point too. >if you want to talk shop about this stuff, post some way to contact you or send an email Oh thank you but no, I work with visuals and computers are cool, but these things make no sense beyond the surface level. The only math I did in school was subtracting from the clock to calculate when math class would end. Keep doing your thing!

>>21614 >vidrel Mmmmm tasty AI flesh.

>>22154 Not sure what "the book of swarm intelligence" is but I have this.

>>22166 Thank you. It's this book.

>>22168 Found it!

>>22191 Im not studying AI or something totally related to the book atm, but I appreciatte your action of remembering to share after the finding.

Does someone have an article about convolutional or MLP neural network which has a good dataset to work and rebuild on? I would like to find one because a teacher of my Computational Inteligence classes want us to make some imporvements on the model of any article.

>>22193 Hey sharing is caring. Anyone else have other books they need?

>>22194 nothing specific, but you might want to look at some of the early work in handwriting recognition. that's sort of the archetypal example when it comes to CNNs and MLPs.

im making a program that can tell mirrors from windows

>>22191 Thank you so much.

Me and some folks are trying to implement a convolutional neural network, based on the article linked here. When trying to fit the model, we get the following error: history=model.fit(X_train, Y_train, validation_data=(X_test, Y_test),epochs=50, batch_size=32) ValueError: Input 0 of layer sequential is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 21] Does anyone have a clue to what happens here?

Can someone point me into some kind of direction with dependency parsing? I pretty much just copying and pasting examples at this point, this gave me a general idea: https://www.youtube.com/watch?v=PVShkZgXznc But I still can't get a solid idea on what this is based on. I've tried something like "I want to book a taxi to the airport and a table at the restaurant" how does table relate to restaurant but taxi doesn't relate to airport?

>>22411 Try using displacy, it gives you a nice graph of dependency trees that you can visualize for better understanding.

>>22320 you need to reshape it, fit method expects 3d tensors. Something like that X = np.reshape(X, (X.shape[0], 1, X.shape[1])) read mor e https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc

>>22411 check out GloVe

what are your thoughts on huggingface?

I don't know if I should frame this as an ML problem and I want to start off saying I have no experience in ML. Basically I want to create a custom text classification model that can divide text into a limited set of categories (9). I also want to see the probability of a match not just general accuracy. Please help.

>>25294 do you have labelled data for this problem? otherwise you would use knn or something like that to cluster the data

>>25208 based as fuarrrk. Can load financial model with pretrained weights and recognize sentiment with decent accuracy, all in few lines of code and easy to use

>>1361 The year is now 2021. Norvig and Russel's book, "Artifical Intelligence: A Modern Approach" got an update in 2020 that focuses more on machine learning.

pdf rel is my on-topic contribution. but if I want to discuss Forth only, should I start a new thread?

How stupid is my dream of making an AI waifu?

>>29122 Very. Currently models that generate language like GPT-3 (one of the best-known models) are mostly copy-cats. They do not generate good answers they generate an answer that suites to your question based on a underlying distribution. So they are not intelligent they only have a huge memory of things that already had been said on the internet. This is because they used common crawl, a huge dataset with many internet pages in it, to train it plus a few other datasets. If you want to try it just get a beta key, they are open for registration currently.

>>29122 Imagine being rejected by a markov bot

I have a machine learning workshop if anyone is interested https://learnaifromscratch.github.io/ai.html We go through the guy's course who runs Scikit-Learn, and the most advance theoretical course possible by the guy who wrote the book CMU's grad course uses. It's easier than you think, esp the theoretical version the applied version by the scikit-learn guy is much harder because it's applied. Anyone can do this, the theoretical course will teach you all the notation/stats etc., and if they don't I will teach it to you in the notes. God speed anons

Saging, since this is an unrelated elaboration to >>15889 For anyone taking this as truth. It is a daring statement to call "scientific development" hegelian dialectics, because I think the influence of Kuhn is well established in philosophy of science. In this case, more fundamental are paradigms in which the theoretical terminology may be considered incommensurable with the next paradigm. This is not necessarily progress, this is change of belief system where any theory from the old paradigm does not necessarily overlap in the new. The only solid remainder is the observations done, not the theory. H.R. Giger may have meant something else, but as I've interpreted the words, they are very much disputed.

Hello sweethearts, especially data janitors and ML bitches Basically I want to train a classifier to tell if a specif text was written by a certain author After the usually thinks of stemming and removing the stop words, I was about to use the bag of words method to get a vector representing the text and then train a SVM and so on.... But now I want to use Doc2Vec 'cause that way it'll keep the semantics & soykafs of the text. But there not pre trained models except for [1] which is not compatible with modern gensmim So the problem: If I want to train Doc2Vec, is it OK to train it on the same texts that I'll will use to train the classifier later??? I think doing this will introduce a lot of bias, which is baka desu Should I train it on the Wikipedia articles for example??? Also if you have any tips in general of my workflow tell me, 'cause iam a noob. Bye sweetheart, thanks for reading little egg

>>30284 >I want to train a classifier to tell if a specif text was written by a certain author Nice try FSB

>>30299 Do you think a russian agent can write so well in perfidious Albion's idiom??

What is top-k accuracy? Saw this mentioned in a paper as an evaluation metric but I was unfamiliar with it.

>>32018 Top-1 accuracy is the percentage of times the correct label is the top prediction. Similarly, Top-k accuracy is the percentage of time the correct label is in the top k predictions.

Does anyone know what C/C++ libraries are used to speed up things like Pytorch? I want to learn more about them, just to get a better understanding.

>>4187 I'm reading the same book to prep for grad school. Any interest in studying together?

>>29122 2 years ago someone said you are very stupid to try doing it, now you are already too late to the race (Replika and its clones). I wonder: what is the NO BS way of learning AI engineering. Not to get a job or a degree, but to actually practically develop AI software. George Hotz said to just read papers and implement them. I think there's a few steps missing. If you ask Reddit you get the classic responses: a bunch of books and courses. I know people recommend the same if you ask them how to learn programming or game development, but this isn't how I learned either. i learned by doing, not through tutorials. But with AI engineering I don't even know to start... I just know that going through several 1000 page books on statistics isn't the way, just liek i didn't learn geometry, vector maths and C network programming to develop games (though I eventually did have to read up on many such topics as I encountered the problems),

>>36596 neural networks are inherently extremely mathematical rather than practical. comparing it to programming or game dev shows you have no idea what you are talking about. If you want to do more than just apply other peoples research by importing a library you do need many years of learning math. in any case, this type of AI needs mass data that only big companies can gain via malicious data mining. This is the worst field for a solo developer or cypherpunk

>>36596 "AI" is a marketing term. Machine learning and neural networks are quite literally just applied statistics and mathematical optimization. You can be a programmer and use the tech but dont call yourself an "AI engineer" unless you know PhD level math

>>1324 I work on deep learning research, but not in the CS field. Our lab's focus is genomics. Research is mostly about taking existing architectures and modifying them / merging them so that they give good results with genomic data. There's a lot of work put into data augmentation and performance since we don't have nearly the same amount of genomic data as text data for training, most of the time it's vastly less.

does anyone have any papers similar in style to this one?

>>36598 I feel like that's why it's best to just put on sheets and masks and do the dance and get on their good sides before ultimately taking charge and using that vast resource they possess for your own ends. You can find data for these things anywhere, you just have to make different poses to access it. It's like a rain dance! You work a little and the resources ebb and flow to you.

>>36596 Take a look at the ggml / llama.cpp community. There are a bunch of great low-level ML hackers there. Open models and datasets can be downloaded from huggingface to play around with. That said I agree with the other poster that the math is pretty unavoidable. All of this stuff is built on applied statistics and linear algebra. Learning some math IS the no bs way of learning it.

>>2775 https://www.youtube.com/watch?v=Zm95NWd-2Tw

>>36880 >That said I agree with the other poster that the math is pretty unavoidable. All of this stuff is built on applied statistics and linear algebra. Learning some math IS the no bs way of learning it. Can you name some of ways either of those was helpful to you in ML?

>>246 Currently making a machine learning project as my FYP. A hand gesture recognition app that can be used to make macros. So if you gesture a swipe right with your hands you can change your tabs on the browser, gesture a swipe up and you can scroll the page, etc. The usecase I've thought up for now is something like youre cooking, your hands are covered with flour or something, you can just gesture for the music to stop, or to scroll down the recipe. I've done object classification before, classified different playing cards. Did the Hotdog, notHotdog classification from Silicon Valley as well. Coming to my question, while I know how to classify different hand gestures using tensorflow, I'm a bit new to making applications. I figure I'll have to use some Python GUI library like PyQt or tkinter. But how do I integrate the Machine Learning aspect like tensorflow with the GUI? Are there any resources that might help me? Have people here done this before? Pls help.

>>36880 A late thank you for the hint. Lain is rather slow, I hope you understand

>>37022 Most importantly, knowing statistics is helpful in determining whether you actually need ML to solve a problem. Sometimes a linear regression or a T-test tells you everything you really need to know with a lot less effort.

>>246 >Also Forth. The best ED article ever written imo was on that. https://archive.ph/WHCBl

>>36599 >>36598 Not the person you were replying to, but am someone in a similar situation. I really want to break into the A.I. space and so I'm going to begin reading Deep Learning (Adapative Computation and Machine Learning Series) as well as probability for computer scientist courses. I'm currently a student finishing up a year of calculus, and I'll pay extra attention in Linear Algebra. All I've done is play around with APIs and mostly try to program, I really wish I could get my foot in the door more besides just some guy who borrows from APIs that AI researchers/engineers spent time on. Because at that point, I'm just a programmer, as you guys agree.

has anyone had a go at reading The Principles of Deep Learning Theory by Roberts & Yaida? If so, how did you find it?

>>2227 [WARNING OFFTOPIC] privacy is a red herring. internet is not and cannot ever be "private" the whole point of internet is information sharing (that is the reason why the most perfect private computer is the one that has no network connection whatsoever) instead what you really want is not privacy but anonymity on the internet. Since cost of copying information is always zero any form of info-withholding, gatekeeping, pay-walling, copyrighting, privacy law passing, block-chaining will never work. In other words the problem is not that some parties have access to information that you knowingly or unknowingly posted on the net but the fact that this information can always be linked back to you. So the problem then is authorship of information and internet is proof that information has no inherit authorship. Authorship is something added on to it, like a parasite, to extort value of some form (usually monetary) out of it. Thats the discomfort the "NoAI" "artists" feel when they see how users of AIGen software effectively claim that they are the authors of the software's product and as such should be paid for it (and then these noAI people immediately go on and claim they are the REAL authors of all of the images and therefore they are the ones who should be paid. pathetic.)