I've started a programming project for school recently where I'll be making an AI written in C++. I have done projects before, but usually small things, nothing of this size but I do believe I have the knowledge and drive to complete such a task. The only problem is I'm not entirely sure where to start, I've looked into Neural Networks and I understand that they are used to simulate(to some degree) the neurons in the brain, but I have not really found out how to implement them.

TL;DR I've started an AI project and am not sure where to begin

For what it's worth, I'm also officially Tribal's advisor for this. I think the important things to mention here are what you want to use the neural network for (text processing, I believe), what you want the network to do, and what kind of corpus set you'll be using to train and test your network. Can you give us some more details, and/or brainstorm with us?

Yes, the neural network will be used for text processing, and as for the corpus I have not really thought of a set from which it would be derived. It would probably be a combination of IRC logs, news articles, literary material and the like. I was hoping the network would, at it's mature stage, be able to give intellectual responses when engaged in conversation.

In other words, exactly Simms AI . Hopefully the Simms AI v1 tech report helped clarify what a difficult task that is; can you suggest a more realistic goal that would be a simpler to achieve in the amount of time you have left before the end of the semester?

Hmm, I have not really given much thought to actually reaching the goal. My original idea was that I would get as much done as possible and present what I had gotten done, and detailing what has yet to be accomplished and needs to be finished before the final product would be in fully functional order.

Indeed, but I think it would be cool if you could demo something functional. Full sentences are probably a bit too ambitious, for example, something like concept association would still be extremely challenging but possibly something to which to strive.

Well, I do agree that it would be nice to actually be able to be able to demo something, I'm not sure what could be shown.

One thing I always wanted to do was mess with text-processing or associative neural networks combined with some kind of "emotion" tagging, since our memories are strongly emotion-tagged. Being in a particular emotional state tends to enhance accessibility of memories stored while in a similar state and suppress accessibility of memories stored in a disparate emotional state, as far as I can remember.

I was actually looking into emotion not a couple days ago, it's also a topic in my online psychology course which I could probably get some more information from concerning it.

You kinda gave me an idea on what I could demo as well, maybe have a part of the program that would show a live image of connections of conversation etc. Even if the response is not very intellectual, they could see the connections it is making to provide that response.

That's a good idea; visualization is an important part of a project like this, since to the layperson the concepts might be a bit inaccessible. Did you decide on a language and platform for the project, or are you open to suggestions? I would definitely NOT go with the LAMP architecture that Simms1 used.

I'm open to suggestions, but I'm rather hard set in doing it in C++. I was planning on making it as modular as possible so to cut back on compilation times. This could also lead way to on-the-fly upgrades by compiling the new source, running the module, and connect to a manager who would see that it is a newer version of a already running module through some id mechanism. It would send any information it needs to, to the new module and when it's ready take the older module offline allowing for zero down-time upgrades.

Sounds complex! Definitely a good idea for the finished product, but be sure that you're being realistic about the scope of what is going to ensure you retain your sanity for a single-semester project.

Bad choice. You'll waste tons of time debugging your C++ code instead of debugging your algorithms. Lisp, Python, C#, or Java would all be much better choices.

Doing runtime-updating of shared libraries alone is going to be a nightmare. Doing that requires a ton of extra work on your part (shared libraries stop being automatic entirely - you have to lookup every method you want to invoke, huge PIA). Yet on Python something like that would be as trivial as "reload(module)" (assuming your code could persist it's state somehow, of course - pickle would fill that role nicely methinks)

I was going to suggest Python as well, especially if you're going to be doing a lot of string parsing and manipulation. It will also be better if you go my route and play around with replication in some near or far future, but that's a minor detail.

Well, my choice of C++ was purely because it is what I am most familiar with.

Anyway, I think I may have set my goals too high since my deadline has came and gone and all I have to show is my research. I'm not really sure if I could pull something together before the presentation date arrives, but it would be nice. The main thing that has stopped me was that I am still confused on the idea of how to programmatically get an AI to work. I've found a few code examples online, but they really didn't help much. One of them is a program called 'Smart Sweeper' where these little sweepers are suppose to go around a pick up these green dots. They start out moving in circles, but eventually learn to move towards the dots and pick them up. That one uses a feed forward neural network. The other example I was able to find I couldn't get to compile(actually I couldn't get either to compile, but the other had a pre-built executable), but uses Long Short-Term Memory and is suppose to be able to learn reber grammer.
I'm not really sure what I could possibly pull together, but from thinking about it for a little here are some ideas that I came up with, but don't know how small they would be nor really how they would be done:
1. I could maybe make a checkers AI. I think this would probably be the easiest.
2. Character recognition? I'm not really sure how hard this would be, but it seems like it might be kind of easy.
3. I'm kinda out of ideas here :/

1. Surprisingly, a checkers AI is extremely challenging. The AI class at my alma mater gives students a full semester to build a checkers AI with various heuristics, and many still don't have a working project by the end of the semester, let alone a well-playing one.
2. This one is "easy" (ha! as "easy" as any image processing task, namely, not the hell at all) with a Convolutional Neural Network, the same type that I used for my thesis. A standard NN can do a fairly good job, but only if the font doesn't change much, and the size of letters doesn't change much. If you were to restrict it to one size of one font, a small character set (like A-Z, space, 0-9), you could come up with a pretty decent project. You should also use something monospaced, because performing segmentation (figuring out where the letter boundaries are) is another problem in itself.

If Kerm is your adviser, you're in good hands. He advised an independent study I did several years ago that turned out rather well.


I would also suggest Python. Though don't let its awesome string manipulation capabilities fool you into skipping the use of regex where appropriate.

Definitely, regex is always your friend no matter what you're doing (or, to be more precise, for everything except XHTML/HTML markup). And elfprince13, I appreciate your vote of confidence, and hope I can live up to it.

Or to be more precise yet: for every problem that can be solved by a finite state machine. This means nothing requiring stacks or a writable tape. Basically, good for lexicographic analysis (tokenizing), bad for parsing.

Indeed, that's why we have a difference between regular languages and context-free languages. Huzzah for pumping lemmae! Anyway, if Tribal does indeed end up working with some kind of character recognition, all this language nonsense is water under the bridge, or over the dam, or wherever it goes.