.txt to .8xp Converter
» >
» Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
» View previous topic :: View next topic  
ScoutDavid wrote:
Richard Nixon! Now that makes some sense!!!


Now, I may be moving to C#, any recommendations?

Merth's Tokens is C#. I would prefer to make a team with him and help him out, but he refused it already.

So, I can ask him for help on making my own one, right?


Why? What do you hope to gain by switching to C#?
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
Kllrnohj wrote:
ScoutDavid wrote:
Richard Nixon! Now that makes some sense!!!


Now, I may be moving to C#, any recommendations?

Merth's Tokens is C#. I would prefer to make a team with him and help him out, but he refused it already.

So, I can ask him for help on making my own one, right?


Why? What do you hope to gain by switching to C#?
It sounds to me that it's less about what he could gain by switching to C# than about what he could lose by switching away from Python (namely, his frustration with it).

Code:
It sounds to me that it's less about what he could gain by switching to C# than about what he could lose by switching away from Python (namely, his frustration with it).


When I google, I find many programming language converters and compilers made in C# and not many in Python.

I think the amount of information available for this process in C# will make it easier for me to code the program.

Regarding XML, I don't know XML. I promised myself to learn it in Christmas, so as PHP.
ScoutDavid wrote:

Code:
It sounds to me that it's less about what he could gain by switching to C# than about what he could lose by switching away from Python (namely, his frustration with it).


When I google, I find many programming language converters and compilers made in C# and not many in Python.

I think the amount of information available for this process in C# will make it easier for me to code the program.
I worry that your reasoning may be fallacious, because many programming languages have much stricter grammar than TI-BASIC and are thus easier to parse and build an AST from. For example, regex-based Yacc/Bison definitions can be build for C99 that enable relatively painless AST generation/tokenization of C. TI-BASIC is much less clear.

ScoutDavid wrote:
Regarding XML, I don't know XML. I promised myself to learn it in Christmas, so as PHP.
Luckily, XML itself is a near-trivial concept. Using it effectively is not.
KermMartian wrote:
It sounds to me that it's less about what he could gain by switching to C# than about what he could lose by switching away from Python (namely, his frustration with it).


And yet all his frustration so far has been with the concepts, not the language. The concepts won't get easier by moving to a new language - it'll just slow him down since he'll first have to learn the language, then begin struggling with all the same concepts.

ScoutDavid wrote:
When I google, I find many programming language converters and compilers made in C# and not many in Python.

I think the amount of information available for this process in C# will make it easier for me to code the program.


No it won't. It might make it easier to "borrow" others code, but that isn't learning. You don't want language converters or compilers either, what you are trying to build is neither. What you are trying to build is a tokenizer, and Kerm has already tried explaining what you need to do for that.

If you want a full compiler for a well defined syntax, look no further than PLY ( http://www.dabeaz.com/ply/ ), a really good port of Lex and Yacc for Python. But again, this isn't what you want. You've been searching for the wrong things. And again, it is the *CONCEPTS* you need to learn and are struggling with. Looking at code is not really how you learn those.
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
Kllrnohj wrote:
KermMartian wrote:
It sounds to me that it's less about what he could gain by switching to C# than about what he could lose by switching away from Python (namely, his frustration with it).


And yet all his frustration so far has been with the concepts, not the language. The concepts won't get easier by moving to a new language - it'll just slow him down since he'll first have to learn the language, then begin struggling with all the same concepts.
Oh, I know, and I'm sure you know I know. I was trying to explain the logic that I think he was using.

Quote:
ScoutDavid wrote:
When I google, I find many programming language converters and compilers made in C# and not many in Python.

I think the amount of information available for this process in C# will make it easier for me to code the program.


No it won't. It might make it easier to "borrow" others code, but that isn't learning. You don't want language converters or compilers either, what you are trying to build is neither. What you are trying to build is a tokenizer, and Kerm has already tried explaining what you need to do for that.

If you want a full compiler for a well defined syntax, look no further than PLY ( http://www.dabeaz.com/ply/ ), a really good port of Lex and Yacc for Python. But again, this isn't what you want. You've been searching for the wrong things. And again, it is the *CONCEPTS* you need to learn and are struggling with. Looking at code is not really how you learn those.
Although Kllrnohj is going the tough-love route, he's absolutely right, I'm afraid. I think that some of the basic language concepts are tripping you up rather than the specifics of tokenization, unfortunately. If you agree that that might be the case, I'd probably recommend trying some simpler Python (or C#) projects first to help yourself understand concepts of program flow and structure, data types and representations and manipulation, and thinking more like an interpreter (in this case, the Python interpreter or the C# compiler) in terms of how the code that you're writing behaves when executed.
I am comfortable with C# up to a certain point.
I'm not learning it now xD
ScoutDavid wrote:
I am comfortable with C# up to a certain point.
I'm not learning it now xD
Do you find that your knowledge of C# does or does not generally transfer over to Python?
I'm not getting yhour question but I am more comfortable with Python.
ScoutDavid wrote:
I'm not getting yhour question but I am more comfortable with Python.
I'm asking if you find that your C# skills are helping you with Python and vice versa, or if the two seem more distinct to you. Which do you feel more comfortable writing in, or are you equally comfortable with the two?
KermMartian wrote:
ScoutDavid wrote:
I'm not getting yhour question but I am more comfortable with Python.
I'm asking if you find that your C# skills are helping you with Python and vice versa, or if the two seem more distinct to you. Which do you feel more comfortable writing in, or are you equally comfortable with the two?


I am more comfortable writing in Python than in C#.

They are totally distinct, uncomparable.
ScoutDavid wrote:
I am more comfortable writing in Python than in C#.

They are totally distinct, uncomparable.
I completely disagree, if you don't mind. Smile Although C# enforces the object-orientedness a bit more strongly, they share similar concepts of programing flow and structure, object orientation, modules and interfacing, and quite a bit more. I expect that Kllrnohj and Benryves will charge in here and correct me, but that's my general impression, having coded much, much more Python than C#.
KermMartian wrote:
ScoutDavid wrote:
I am more comfortable writing in Python than in C#.

They are totally distinct, uncomparable.
I completely disagree, if you don't mind. Smile Although C# enforces the object-orientedness a bit more strongly, they share similar concepts of programing flow and structure, object orientation, modules and interfacing, and quite a bit more. I expect that Kllrnohj and Benryves will charge in here and correct me, but that's my general impression, having coded much, much more Python than C#.


Yes, C# is object-oriented, you can also do OOP in Python but it's not what it was made for.

However, I don't know which one would be better for a tokenizer
I feel like Python has a lot less "baggage" attached to it, but I think that's probably my personal bias against Microsoft IDEs talking. In all truthfulness they're decent programs, as Benryves has tried to impress upon me many times, but I like the crossplatformness of Python, so that's what I'd go with myself.
The Microsoft IDEs are probably the most powerful developer tools available.
But yeah, I suppose the not being cross-platform is a problem.
Shaun
KermMartian wrote:
I completely disagree, if you don't mind. Smile Although C# enforces the object-orientedness a bit more strongly, they share similar concepts of programing flow and structure, object orientation, modules and interfacing, and quite a bit more. I expect that Kllrnohj and Benryves will charge in here and correct me, but that's my general impression, having coded much, much more Python than C#.


I more or less agree with you. They are absolutely comparable, as neither is something totally different like Python vs Haskell. Most languages are comparable.

Quote:
Yes, C# is object-oriented, you can also do OOP in Python but it's not what it was made for.


Not true at all. Python was absolutely built to be OO.

Quote:
However, I don't know which one would be better for a tokenizer


python will be "better" (aka, easier)
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
I second that; Python will be the easier method, and I feel that if you're running into difficulties with Python, perhaps conceptual difficulties with the language rather than application-specific difficulties are the culprit.
KermMartian wrote:
I second that; Python will be the easier method, and I feel that if you're running into difficulties with Python, perhaps conceptual difficulties with the language rather than application-specific difficulties are the culprit.


Outside of the dict that holds all the tokens, the tokenizer itself shouldn't be more than 50 lines (probably less than 20 tbh)
There are only two kinds of programming languages: those people always bitch about and those nobody uses. (Bjarne Stroustrup)
Kllrnohj wrote:
KermMartian wrote:
I second that; Python will be the easier method, and I feel that if you're running into difficulties with Python, perhaps conceptual difficulties with the language rather than application-specific difficulties are the culprit.


Outside of the dict that holds all the tokens, the tokenizer itself shouldn't be more than 50 lines (probably less than 20 tbh)
Indeed, I believe that mine was around the 20-line mark. It's a relatively straightforward process once you get your mind around it.

Code:
###############################################################
# Name:                  Tokenizer.py
# Purpose:       Abstract tokenizer
# Author:                Luca Allulli (webmaster@roma21.it)
# Created:       2009-01-31
# Copyright: Luca Allulli (http://www.roma21.it/songpress)
# License:       GNU GPL v2
##############################################################

import re

class Token(object):
        def __init__(self, token, start, end, content):
                object.__init__(self)
                self.token = token
                self.start = start
                self.end = end
                self.content = content
               
        def __str__(self):
                return '%s, %d, %d, %s' % (str(self.token), self.start, self.end, self.content)
               
class TokenType(object):
        def __init__(self, r, name):
                object.__init__(self)
                self.r = re.compile(r)
                self.name = name
               
        def __str__(self):
                return self.name
       
               
class Tokenizer(object):
        def __init__(self, l):
                object.__init__(self)
                self.line = l
                self.pos = 0
                self.state = self.normal
               
        def __iter__(self):
                return self
               
        def next(self):
                if self.pos >= len(self.line):
                        #End of line
                        raise StopIteration
                for t in self.transition[self.state]:
                        #print "Trying: %s" % (t[0],)
                        m = t[0].r.match(self.line, self.pos)
                        if m != None:
                                #print("Matched!")
                                break
                if m == None:
                        return None                             
                tok = Token(t[0], self.pos, m.end(0), m.group(0))
                #print (type(m.group(0)), m.group(0))
                self.state = t[1]
                #print("Found %s" % (tok,))
                self.pos = m.end(0)
                return tok


I just found this.

THis is supposed to work for all kinds of tokenizers. I probably need to start from scratch because I want a TI83+ tokenizer, right?
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
.txt to .8xp Converter
» Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
» View previous topic :: View next topic  
Page 7 of 8
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement