Discussion:
comma-less lists and tuples
(too old to reply)
unknown
2006-09-20 14:17:30 UTC
Permalink
Hi,

When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a delimiter in lists
and tuples in place of commas. Consider this small example:

{select, [foo, bar, baz], {from, [table1, table2]}, {where, {xyz, '=',
{'+', [1,2,3,4]}}}}

If commas were optional, this example could be written as

{select [foo, bar, baz] {from [table1, table2]} {where {xyz '=' {'+'
[1 2 3 4]}}}}

I think the second line is more readable, and it's also easier to
write. In other DSELs, the difference may be even more substantial.

What do you think? Am I missing something?

Best regards,
Yariv
unknown
2006-09-20 14:35:12 UTC
Permalink
Welcome back to LISP? :)
Post by unknown
Hi,
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a delimiter in lists
{select, [foo, bar, baz], {from, [table1, table2]}, {where, {xyz, '=',
{'+', [1,2,3,4]}}}}
If commas were optional, this example could be written as
{select [foo, bar, baz] {from [table1, table2]} {where {xyz '=' {'+'
[1 2 3 4]}}}}
I think the second line is more readable, and it's also easier to
write. In other DSELs, the difference may be even more substantial.
What do you think? Am I missing something?
Best regards,
Yariv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20060920/ca545ba6/attachment.html>
unknown
2006-09-20 14:40:42 UTC
Permalink
Post by unknown
Hi,
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a delimiter in lists
would such a change create a problem when atoms look like this:
'atom with space'?


bengt
--
EPO guidelines 1978: "If the contribution to the known art resides
solely in a computer program then the subject matter is not
patentable in whatever manner it may be presented in the claims."
unknown
2006-09-20 14:48:22 UTC
Permalink
Post by unknown
Post by unknown
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a
delimiter in lists
'atom with space'?
I don't see why if we can have 'atom,with,commas' already.

*Erik.
unknown
2006-09-20 15:08:12 UTC
Permalink
Post by unknown
Post by unknown
Post by unknown
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a
delimiter in lists
'atom with space'?
I don't see why if we can have 'atom,with,commas' already.
While we are discussing commas, it would be extremely nice to allow a
trailing comma in lists and tuples:

[1,2,3,]
{1,2,3,}

This is very useful during development, it provides for less source editing
with no assumption that datastructures are cast in stone.

Many languages support that already, including gcc variant of C.

Pretty please?

\Anton.
--
We're going for 'working' here. 'clean' is for people with skills...
-- Flemming Jacobsen
unknown
2006-09-20 15:12:12 UTC
Permalink
personally, i think it would just make the language more confusing if
you added this, especially to beginners.

Mog
unknown
2006-09-20 15:42:48 UTC
Permalink
Post by unknown
Post by unknown
Post by unknown
Post by unknown
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a
delimiter in lists
'atom with space'?
I don't see why if we can have 'atom,with,commas' already.
While we are discussing commas, it would be extremely nice to allow a
[1,2,3,]
{1,2,3,}
This is very useful during development, it provides for less source editing
with no assumption that datastructures are cast in stone.
Many languages support that already, including gcc variant of C.
Pretty please?
Are these really features worth debating at this time? Aren't there more
important considerations regarding erlang than the ability to omit (or add
extra) commas in lists?

I don't mean to rain on the parade but this seems somewhat unimportant to me.

-Rick
unknown
2006-09-20 17:05:30 UTC
Permalink
Post by unknown
Post by unknown
Post by unknown
Post by unknown
Post by unknown
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a
delimiter in lists
'atom with space'?
I don't see why if we can have 'atom,with,commas' already.
While we are discussing commas, it would be extremely nice to allow a
[1,2,3,]
{1,2,3,}
This is very useful during development, it provides for less source editing
with no assumption that datastructures are cast in stone.
Many languages support that already, including gcc variant of C.
Pretty please?
Are these really features worth debating at this time? Aren't there more
important considerations regarding erlang than the ability to omit (or add
extra) commas in lists?
I don't mean to rain on the parade but this seems somewhat unimportant to me.
It certainly helps to have syntax features that make it easier to
write correct code. Allowing extra commas in lists and tuples would be
a small and easy win for usability.

However, adding a feature like this encourages people to write code
that isn't backwards compatible, so perhaps it's not worth doing.

-bob
unknown
2006-09-20 15:19:59 UTC
Permalink
Post by unknown
personally, i think it would just make the language more
confusing if you added this, especially to beginners.
I did not write that I was in favor of the comma-less lists and tuples,
just that it would not be a problem for atoms, as suggested.
I do not see it as an improvement either.

*Erik.
unknown
2006-09-20 15:20:50 UTC
Permalink
I think you have been staring at your monitor too long ;-).
I agree it does look a bit nicer. But relative to the rest of the
term, its not a big gain.

I have been thinking of DSEL stuff as well lately. I was looking at
Joe's ML9 and trying to see whether using that or prettier term
structure is the better choice for certain types of metadata.

Boston is a nice town and its going to get cold soon...go outside and
stretch your eyes for a few minutes ;-)
BTW, I am enjoying erlyDB...great work!!!
ke han
Post by unknown
Hi,
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a delimiter in lists
{select, [foo, bar, baz], {from, [table1, table2]}, {where, {xyz, '=',
{'+', [1,2,3,4]}}}}
If commas were optional, this example could be written as
{select [foo, bar, baz] {from [table1, table2]} {where {xyz '=' {'+'
[1 2 3 4]}}}}
I think the second line is more readable, and it's also easier to
write. In other DSELs, the difference may be even more substantial.
What do you think? Am I missing something?
Best regards,
Yariv
_______________________________________________
erlang-questions mailing list
erlang-questions
http://www.erlang.org/mailman/listinfo/erlang-questions
unknown
2006-09-20 15:46:54 UTC
Permalink
Post by unknown
I think you have been staring at your monitor too long ;-).
I agree it does look a bit nicer. But relative to the rest of the
term, its not a big gain.
I agree it's not a big gain, but in the Which Is A Better Language For
DSEL pissing contest between Erlang and Lisp, it would at least score
Erlang a couple of points :)
Post by unknown
I have been thinking of DSEL stuff as well lately. I was looking at
Joe's ML9 and trying to see whether using that or prettier term
structure is the better choice for certain types of metadata.
I think the combination of atoms, tuples, lists and pattern matching
makes Erlang a very good host language for DSELs. Maybe not as good as
Lisp, but definitely better than most. Pattern matching especially
makes the implementation of DSELs very intuitive, as I learned when
implementing ErlSQL.

It's funny, because initially ErlSQL was just a bunch of functions
called 'select()', 'insert()', 'update(),' etc, that took a list of
arguments for the body of the query. Only after I made enough
functions, I realized that you can map them to an abstract description
that looks very similar to SQL. Starting out, I didn't realize that
SQL can be mapped so intuitively to Erlang, not to mention that Erlang
is a good host for DSELs :)
Post by unknown
Boston is a nice town and its going to get cold soon...go outside and
stretch your eyes for a few minutes ;-)
BTW, I am enjoying erlyDB...great work!!!
The first version was just a proof of concept. The "real" ErlyDB is
coming soon :)

Best,
Yariv
unknown
2006-09-20 15:47:21 UTC
Permalink
(Sorry about top-posting, I'm trying out Yahoo's new email.)


Smalltalk has a similar notation, something like "select: [foo bar baz] from: [tab1, tab2] where: [xyz '=' {'+', [1 2 3 4]}]"'

which would invoke (in Erlang syntax):

'select:from:where'(A1, A2, A3) -> ...

I thought that was a pretty neat idea, but I'm not sure how to integrate it with Erlang. However, to get back to the actual proposal: in Erlang as we know it, the comma-less notation is ambiguous, e.g.

{X andalso Y} = {(X andalso Y)} or {X, 'andalso', Y}?

So that would have to be fixed first. Anyway, those who are interested could have a go at erl_parse.yrl and see whether it can be done.

Best,
Thomas

----- Original Message ----
From: Yariv Sadan <yarivvv>
To: erlang-questions <erlang-questions>
Sent: Wednesday, September 20, 2006 4:17:30 PM
Subject: [erlang-questions] comma-less lists and tuples

Hi,

When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a delimiter in lists
and tuples in place of commas. Consider this small example:

{select, [foo, bar, baz], {from, [table1, table2]}, {where, {xyz, '=',
{'+', [1,2,3,4]}}}}

If commas were optional, this example could be written as

{select [foo, bar, baz] {from [table1, table2]} {where {xyz '=' {'+'
[1 2 3 4]}}}}

I think the second line is more readable, and it's also easier to
write. In other DSELs, the difference may be even more substantial.

What do you think? Am I missing something?

Best regards,
Yariv
unknown
2006-09-20 15:58:00 UTC
Permalink
Post by unknown
in Erlang as we know it, the comma-less notation is
ambiguous, e.g.
{X andalso Y} = {(X andalso Y)} or {X, 'andalso', Y}?
So that would have to be fixed first. Anyway, those who are
interested could have a go at erl_parse.yrl and see whether it can be
done.
I think you'll find this to be the main snag:

{F (1+X) Y} = {F(1+X), Y} or {F, 1+X, Y}

/Richard
unknown
2006-09-20 18:02:26 UTC
Permalink
Hi,
Post by unknown
When working with ErlSQL, I came to the opinion that in some cases,
Erlang would benefit from allowing whitespace as a delimiter in lists
{select, [foo, bar, baz], {from, [table1, table2]}, {where, {xyz, '=',
{'+', [1,2,3,4]}}}}
If commas were optional, this example could be written as
{select [foo, bar, baz] {from [table1, table2]} {where {xyz '=' {'+'
[1 2 3 4]}}}}
I think the second line is more readable, and it's also easier to
write. In other DSELs, the difference may be even more substantial.
IMHO, this is a case for a real DSL - I mean, what's the point in
contriving a DSL's syntax to follow Erlang's or extend Erlang syntax
to match arbitrary DSLs? Maybe other DSLs will require lists delimited
with {} and tuples with [], for example...

Better to use a syntax that's matching the DSL and write a parser for
it, I think.

In this particuar case, I'd either go with full sql (it's at least
standard, people already know it) or find an Erlang equivalent that
isn't following sql that closely.

Just my 2c.

regards,
Vlad
unknown
2006-09-20 18:22:38 UTC
Permalink
Hi Vlad,
Post by unknown
IMHO, this is a case for a real DSL - I mean, what's the point in
contriving a DSL's syntax to follow Erlang's or extend Erlang syntax
to match arbitrary DSLs? Maybe other DSLs will require lists delimited
with {} and tuples with [], for example...
I don't think Erlang's syntax should be extended to match arbitrary
DSLs... however, I do think that allowing whitespace to be treated as
a delimiter (if it's possible and it wouldn't break any existing code)
would make the existing Erlang syntax, which is already quite friendly
for a variety of DSLs (of which ErlSQL is just one example), even more
DSL-friendly.
Post by unknown
Better to use a syntax that's matching the DSL and write a parser for
it, I think.
That's an option, but then your build process becomes more cumbersome.
On the other hand, if you stay in Erlang, you are more limited by the
constraints of Erlang's syntax, but this is probably good enough for a
large number of DSLs. For the more advanced ones, a special parser may
be required.
Post by unknown
In this particuar case, I'd either go with full sql (it's at least
standard, people already know it) or find an Erlang equivalent that
isn't following sql that closely.
The main benefit of ErlSQL is that it makes programmatic generation of
queries in Erlang easy because you work with the semantic elements of
SQL directly in Erlang rather than doing string concatenation. The
examples I've shown have a make a literal translation to SQL, which
may not seem so impressive, but examples that involve more
programmatic SQL generation look far better with ErlSQL than with
string-concatenation.

Regards,
Yariv
unknown
2006-09-20 18:58:04 UTC
Permalink
Post by unknown
I don't think Erlang's syntax should be extended to match arbitrary
DSLs... however, I do think that allowing whitespace to be treated as
a delimiter (if it's possible and it wouldn't break any existing code)
would make the existing Erlang syntax, which is already quite friendly
for a variety of DSLs (of which ErlSQL is just one example), even more
DSL-friendly.
Yes, maybe. What I meant was that there may be other varieties of DSLs
that would benefit from other syntax changes. I agree the list/tuple
example was not realistic, but once starting on that path it's hard to
stop.
Post by unknown
Post by unknown
In this particuar case, I'd either go with full sql (it's at least
standard, people already know it) or find an Erlang equivalent that
isn't following sql that closely.
The main benefit of ErlSQL is that it makes programmatic generation of
queries in Erlang easy because you work with the semantic elements of
SQL directly in Erlang rather than doing string concatenation.
Yes, I know, you are working with is the syntax tree of the sql
statements. Syntax trees need not (and in practice aren't) bear any
resemblance with the textual form of the original. I will argue that
it is even counterproductive - if the syntax tree form is meant to be
written by a human. Why write
{select, [bob], {from, [person]}}
instead of
select bob from person
?

If the tree form is just an internal form that only other programs
will manipulate, then the syntax of the data representing the tree
becomes irrelevant. Likewise the optimizations you started to
introduce (prefix operators).
Post by unknown
The examples I've shown have a make a literal translation to SQL, which
may not seem so impressive, but examples that involve more
programmatic SQL generation look far better with ErlSQL than with
string-concatenation.
Wonderful. The question is then: is it meaningful to change Erlang
syntax just so that a particular class of programs just _look far
better_?

If the goal is to make sql-building code more readable, there is
nothing that beats writing sql code, IMHO. The metaprogramming syntax
I am working on is still in the future (and is still a change to the
Erlang language), but even today you can do
something like

Column = "bob",
Table = "person",
sql:statement("select $Column from $Table")

where sql:statement() would be processed by an adequate parse transform to

["select ",sql:escape(Column)," from ", sql:escape(Table)]

and if you need to, it could even parse it and return the syntax tree as above.

I may be biased because I don't work that much with sql, but to me it
is very readable and if I were a sql developer I wouldn't have to
learn a new way to express sql statements.

I hope I'm not sounding like I am putting you down, Yariv. I agree
very much with the idea behind ErlSql, it is just the form that
doesn't feel very comfortable to me.

regards,
Vlad
unknown
2006-09-21 01:33:29 UTC
Permalink
"Yariv Sadan" <yarivvv> wrote:
When working with ErlSQL, I came to the opinion that in some
cases, Erlang would benefit from allowing whitespace as a
delimiter in lists and tuples in place of commas.
Consider this small example:

{select, [foo, bar, baz], {from, [table1, table2]}, {where,
{xyz, '=', {'+', [1,2,3,4]}}}}

The thing that strikes *me* most forcibly about that is the
quantity of noise words. Coding something like this in Prolog,
I would use

select(Selection, Source[, Restriction[, Grouping]])

e.g.

select([foo,bar,baz],
table1 * table2,
xyz = 1+2+3+4)

and this would work rather nicely because Prolog predicate arguments
are not (except in some very exceptional cases) evaluated (and even
then, they are notionally evaluated by the callER, not the callEE).

I note that in SQL 92, the "from" part is not a simple list of tables,
but an expression, so I would be inclined to write

{select,
[foo,bar,baz],
{cross,table1,table2},
{=,xyz,{'+',1,2,3,4}}}

where the general form for an expression is
expression ::= number
| string
| atom
| '{' operator {',' expression}* '}'

operator ::= atom

and we have

'{' 'select' ','
list(expression),
expression
[',' expression % where
[',' list(expression) % group by
[',' expression % having
]]] '}'


If commas were optional, this example could be written as

{select [foo, bar, baz] {from [table1, table2]}
{where {xyz '=' {'+' [1 2 3 4]}}}}

I think the second line is more readable, and it's also easier
to write.

Erlang syntax owes much to Strand88 (whatever happened to Strand88,
anyway?) and Prolog. Prolog has user-defined operators, so this could
not work in Prolog. Erlang doesn't have user-defined operators, but
they would often be nice, and other functional languages like Haskell,
Clean, SML, and CAML have them. It would be sad to adopt a feature
which made user-defined operators unusable now and forever.

One obvious question is whether the Lispy form (don't get me wrong,
Lispy syntax is GREAT, but you have to be consistent about it, and
Lisp, of course, _is_) is easier to write *correctly*. Maybe it is,
but it _looks_ error-prone.

In other DSELs, the difference may be even more
substantial.

We can confidently expect each new DSEL to stress *some* aspect of
Erlang syntax; we *can't* expect them to all stress the *same* aspect.
In particular, I would dumbfounded if there were no DSELs for which
user-defined operators and unevaluated expressions were not extremely
convenient. Come to think of it, SQL is one of them.

By *far* the most readable and writable way to write SQL queries
in Erlang would be something like

@sql[ select foo, bar, baz
from table1, table1
where xyz = 1+2+3+4 ]

where
'@' <identifier> <left bracket>
<bracket-balanced token sequence>
<right bracket>

<left/right bracket> ::= (/) | [/] | {/} | <</>>

would take a sequence of tokens and pass it off to
<identifier>_parser:parse(Tokens)

Within the bracket-balanced token sequence,

'@' '=' <Erlang primary>

would let you embed Erlang expressions (which would be passed as
some kind of token).

Hmm. Wait. Let's add a new declaration

-embedded_syntax(<syntax identifier> , <parser module name>).

to go near the top of a file, so you'd have

-embedded_syntax(sql, my_sql_parser).

and then you could have

Query = @sql{ select foo, bar, ugh from @=Table1, @=Table2
where xyz = @=XYZ }


What do you think? Am I missing something?

Generality.
unknown
2006-09-21 06:56:43 UTC
Permalink
Post by unknown
By *far* the most readable and writable way to write SQL queries
in Erlang would be something like
@sql[ select foo, bar, baz
from table1, table1
where xyz = 1+2+3+4 ]
/snip/

Hey, this is exactly what I suggested a while ago and what I am
working on (but it's not going forward as fast as I'd want) :-) Well,
with a different syntax...

I was preparing something for the EUC, but since I won't be able to
attend I will make available my thoughts on the matter of
metaprogramming in Erlang so that everybody can provide input. I feel
there's a lot of potential here.
Post by unknown
Post by unknown
What do you think? Am I missing something?
Generality.
To put it another way, putting an Erlang syntax around sql statements
(*) is just a way to make the programmer write extra code in order to
make things easier for the compiler. Things should go the other way
around, don't they?

(*) especially if it's basically the same statement, but with added {}
and commas and with some parts enclosed in single quotes

best regards,
Vlad
unknown
2006-09-21 01:45:27 UTC
Permalink
Thomas Lindgren <thomasl_erlang> wrote:
Smalltalk has a similar notation, something like
"select: [foo bar baz] from: [tab1, tab2]
where: [xyz '=' {'+', [1 2 3 4]}]"'

As a Smalltalk user (still haven't finished my Smalltalk compiler;
too busy supervising students) who frequently consults the standard
and peers at the compiler's workings from time to time, I can tell
you that Smalltalk has no such syntax.

Smalltalk array LITERALS don't have commas, so

SQL select: #(foo bar baz)
from: #(tab1 tab2)
where: #(= xyz (+ 1 2 3 4))

would be entirely possible. However, there is no Smalltalk equivalent
of backquote. Suppose we wanted the name "bar", the name "tab2", and
the number "3" to be replaced by the values of Smalltalk variables.

In standard Smalltalk, you would have to write

SQL select: (Array with: #foo with: bar with: #baz)
from: (Array with: #tab1 with: tab2)
where: (Array with: #= with: #xyz with:
#(+ 1 2) , (Array with: three) , #(4))

where , is the infix concatenation operator.
Squeak Smalltalk has an "array literal with all elements evaluated"
syntax, using which we could write

SQL select: {#foo. bar. #baz}
from: {#tab1. tab2}
where: {#=. #xyz. {#+. 1. 2. three. 4}}

Squeak's '{' ... '.' ... '}' syntax is pretty much equivalent to
Erlang's '{' ... ',' ... '}' syntax, except for using full stops
instead of commas.

Actually, *native* Smalltalk style for really querying a data base
would be something like this:

((table1 cross: table2)
select: [:tuple | 1+2+3+4 = tuple xyz])
collect: [:tuple | {tuple foo. tuple bar. tuple ugh}]

The only languages in which this kind of thing is really easy are
the Lisp-family languages.
unknown
2006-09-21 08:16:22 UTC
Permalink
Yes; my example wasn't Smalltalk, just mildly "similar". The purpose was to sketch the "foo:bar:baz" notation; sorry if this was not clear.

Best,
Thomas

----- Original Message ----
From: Richard A. O'Keefe <ok>
To: erlang-questions
Sent: Thursday, September 21, 2006 3:45:27 AM
Subject: Re: [erlang-questions] comma-less lists and tuples

Thomas Lindgren <thomasl_erlang> wrote:
Smalltalk has a similar notation, something like
"select: [foo bar baz] from: [tab1, tab2]
where: [xyz '=' {'+', [1 2 3 4]}]"'

As a Smalltalk user (still haven't finished my Smalltalk compiler;
too busy supervising students) who frequently consults the standard
and peers at the compiler's workings from time to time, I can tell
you that Smalltalk has no such syntax.
/.../
unknown
2006-09-21 09:16:09 UTC
Permalink
Post by unknown
Post by unknown
What do you think? Am I missing something?
Generality.
To put it another way, putting an Erlang syntax around sql
statements (*) is just a way to make the programmer write
extra code in order to make things easier for the compiler.
Things should go the other way around, don't they?
Do you mean teaching the compiler to parse SQL, or to leave
it as is and letting the compiler remain ignorant? ;-)

One purpose of making things easier for the compiler is
of course that the compiler can then _help you_ get it
right. This should then be a metric for evaluating any
special syntax: do we actually gain something by forcing
the programmer to write quasi-SQL rather than SQL directly?

(Despite the obvious problem with SQL that it is by no means
easy to parse - in part because it isn't case sensitive.)
Post by unknown
(*) especially if it's basically the same statement,
but with added {} and commas and with some parts
enclosed in single quotes
Enhanced readability is, I think, a secondary issue, but
also very important. But inventing a new syntax may then
confuse the reader unless it actually serves to make the
statement more intuitive.

BR,
Ulf W
unknown
2006-09-21 09:46:43 UTC
Permalink
Post by unknown
Post by unknown
To put it another way, putting an Erlang syntax around sql
statements (*) is just a way to make the programmer write
extra code in order to make things easier for the compiler.
Things should go the other way around, don't they?
Hi,

What I mean is that IMHO if one wants a structured representation of
something in textual form, the best way is to use a parser. As the
lazy programmer I am, I don't want to do manually do the parser's
work, while at the same time making the representation harder to read
and a nightmare to maintain...
Post by unknown
Do you mean teaching the compiler to parse SQL, or to leave
it as is and letting the compiler remain ignorant? ;-)
How could the Erlang compiler help me write better SQL statements?
Yes, if I write quasi-SQL, I get some syntax checking, but I still can
write stupid things like
{select, {where, {x, '=', 3}}, foo, {from, [table]}}
which is a correct Erlang term, but not correct quasi-SQL.

In the particular case of SQL, we can also skip any parsing and do
fine with strings - especially since sql is not only hard to parse,
but there are many different dialects too. I'd rather let the
database's parser take care of that.

I don't really see what's wrong with concatenating strings :-)
Escaping parts that come from outside needs to be done even with
quasi-sql. Using a parse transform as per my previous example would
make it even almost as readable as pure sql.
Post by unknown
This should then be a metric for evaluating any
special syntax: do we actually gain something by forcing
the programmer to write quasi-SQL rather than SQL directly?
Precisely. If it isn't obvious already, my opinion is that there is no
gain in completely erlang-izing the sql language. But that's just me.

best regards,
Vlad
unknown
2006-09-21 13:47:45 UTC
Permalink
Hi,
Post by unknown
What I mean is that IMHO if one wants a structured representation of
something in textual form, the best way is to use a parser. As the
lazy programmer I am, I don't want to do manually do the parser's
work, while at the same time making the representation harder to read
and a nightmare to maintain...
Post by unknown
Do you mean teaching the compiler to parse SQL, or to leave
it as is and letting the compiler remain ignorant? ;-)
How could the Erlang compiler help me write better SQL statements?
Yes, if I write quasi-SQL, I get some syntax checking, but I still can
write stupid things like
{select, {where, {x, '=', 3}}, foo, {from, [table]}}
which is a correct Erlang term, but not correct quasi-SQL.
In the particular case of SQL, we can also skip any parsing and do
fine with strings - especially since sql is not only hard to parse,
but there are many different dialects too. I'd rather let the
database's parser take care of that.
I don't really see what's wrong with concatenating strings :-)
Escaping parts that come from outside needs to be done even with
quasi-sql. Using a parse transform as per my previous example would
make it even almost as readable as pure sql.
Post by unknown
This should then be a metric for evaluating any
special syntax: do we actually gain something by forcing
the programmer to write quasi-SQL rather than SQL directly?
Precisely. If it isn't obvious already, my opinion is that there is no
gain in completely erlang-izing the sql language. But that's just me.
The first reason to use the "quasi-SQL" for is that it *guarantees*
that you will not expose yourself to SQL injection attacks, whereas
string contcatentation does not. It's like the difference between
coding in a high level language and coding in C. Can you avoid buffer
overflow attacks when writing in C? Sure. But for some reason many
smart people have a hard time avoiding them, which is one reason that
many projects avoid C/C++. Even if you're really smart and you were
able to prevent 99.9% of all buffer overflows in a large codebase,
that single one you didn't prevent can cost you a fortune. Plus, some
programmers on your team may not be as skilled as you in avoiding such
pitfalls. Using a tool that always prevents them can save you a lot of
pain down the road.

Users of ErlyDB and the MySQL driver have asked me, "Your examples use
(some) SQL directly, which makes you volnerable to SQL injection
attacks. How can I prevent them?" That's one of the primary reasons I
made ErlSQL: I wanted to tell them -- "here, use this." And it had to
be easy to use, to look like SQL, and to express a very large subset
of SQL because otherwise they would (legitimately) complain that it's
not intuitive to learn and/or that doesn't do everything they want it
to do.


For the second point: I have said a few times that ErlSQL code looks
much better than string concatenation code when it does a little more
than just literal translation. I'll try to drive the point home with
an example taken from ErlyDB, which makes the query for getting
related records in a many-to-many relation.

Here's the string concatention way:

make_get_related_many_to_many_query(OtherModule, JoinTable, Obj, WhereClause,
ExtraClause) ->
OtherTableStr = atom_to_list(OtherModule),
ThisTableStr = get_module_str(Obj),
JoinTableStr = atom_to_list(JoinTable),
WhereClause1 = make_nested_where_clause(WhereClause),
FieldStrs = make_str_for_fields(OtherModule),
Query =
lists:append(
["SELECT ", FieldStrs, " FROM ",
OtherTableStr, ",", JoinTableStr,
" WHERE (", OtherTableStr,
".id = ", JoinTableStr,
".", OtherTableStr, "_id AND ",
JoinTableStr, ".", ThisTableStr, "_id=",
get_id_str(Obj), ")", WhereClause1, ExtraClause]),
Query.

make_str_for_fields(Module) ->
TableStr = atom_to_list(Module),
{Result, _} =
lists:foldl(
fun(Field, {Str, IsFirst}) ->
FieldName = TableStr ++ "." ++ atom_to_list(Field),
Str1 =
if
IsFirst ->
FieldName;
true ->
FieldName ++ ","
end,
{Str1 ++ Str, false}
end, {"", true}, lists:reverse(Module:fields())),
Result.

make_nested_where_clause(WhereClause) ->
case WhereClause of
undefined ->
" ";
_ ->
" AND (" ++ WhereClause ++ ") "
end.



Here's the ErlSQL way:

make_get_related_many_to_many_query(OtherModule, JoinTable, Obj,
WhereClause, ExtraClause) ->
{select, OtherModule:shown_fields(),
{from, [OtherModule:table(), JoinTable]},
{where,
{{{OtherModule, id},'=',
{JoinTable, get_id_field(OtherModule)}},
'and',
{{JoinTable, get_id_field(Obj)}, '=', get_id(Obj)}},
'and', WhereClause)},
ExtraClause}.


Which do you think is more readable? Which is more writable?

Even putting readability aside, when the Mnesia driver is passed this
structure, it doesn't have to parse anything. It can pattern-match on
the queries that it knows it can handle and breaks on the rest. That's
a big win in simplifying the code.

Best regards,
Yariv
unknown
2006-09-21 18:54:39 UTC
Permalink
Hi!
Post by unknown
The first reason to use the "quasi-SQL" for is that it *guarantees*
that you will not expose yourself to SQL injection attacks, whereas
string contcatentation does not.
Yes, naive string concatenation doesn't. But I guess I didn't express
myself clearly enough: it is not this simple string concatenation that
I compare with ErlSql, but a similar library with the single
difference that the API uses string-sql instead of erlang-sql.

The comparison you make is unfair, because for example
make_str_for_fields/1 is a library function. I bet ten pints of beer
that OtherModule:shown_fields() does something very similar.

So how would this alternative library look like? Taking your example
and without any parse transformations or other non-standard
functionality, I would like to see something like

make_get_related_many_to_many_query(OtherModule, JoinTable, Obj,
WhereClause, ExtraClause) ->
sql:build("select %L from %L "
" where (%s.id=%s.%s "
" and %s.%s= %s) "
" and (%s) "
" %s",
[OtherModule:shown_fields(), [OtherModule:table(), JoinTable],
OtherModule, JoinTable, get_id_field(OtherModule),
JoinTable, get_id_field(Obj), get_id(Obj),
WhereClause,
ExtraClause]).

This can be cleaned up a little (and your erlsql example too) by using
some local variables. %L says the arg is a list of parameters.

If we use a parse transform, the it may look even prettier (in my
eyes, at least)

make_get_related_many_to_many_query(OtherModule, JoinTable, Obj,
WhereClause, ExtraClause) ->
sql:build("select %%OtherModule:shown_fields() "
" from %%[OtherModule:table(), JoinTable] "
" where
(%OtherModule.id=%JoinTable.%get_id_field(OtherModule) "
" and %JoinTable.%get_id_field(Obj)=%get_id(Obj)) "
" and (%WhereClause) "
" %ExtraClause").

What sql:build does behind the scenes is an implementation detail. It
may put together strings (like io:format) or it may parse the sql,
build a quasi-sql representation and work with it. As a user I don't
care, as long as it does what it is supposed to do. I don't have to
keep counting nested {}s and []s and to learn what in fact is a new
query language (you already want to introduce new ways to write
things).

BTW, how do you write 'join' statements? The natural way for an erlang
data structure would be {join, {select,...}, {select,...}}, but that
doesn't follow the way the sql looks like.

Having a new query language isn't a bad thing, but then it's not sql.

best regards,
Vlad
unknown
2006-09-21 19:39:14 UTC
Permalink
(Vlad, sorry for sending this twice -- I forgot to CC the list)
Post by unknown
Yes, naive string concatenation doesn't. But I guess I didn't express
myself clearly enough: it is not this simple string concatenation that
I compare with ErlSql, but a similar library with the single
difference that the API uses string-sql instead of erlang-sql.
The comparison you make is unfair, because for example
make_str_for_fields/1 is a library function. I bet ten pints of beer
that OtherModule:shown_fields() does something very similar.
You owe me ten pints -- that function returns a list of atoms :)
- Show quoted text -
Post by unknown
So how would this alternative library look like? Taking your example
and without any parse transformations or other non-standard
functionality, I would like to see something like
make_get_related_many_to_many_query(OtherModule, JoinTable, Obj,
WhereClause, ExtraClause) ->
sql:build("select %L from %L "
" where (%s.id=%s.%s "
" and %s.%s= %s) "
" and (%s) "
" %s",
[OtherModule:shown_fields(), [OtherModule:table(), JoinTable],
OtherModule, JoinTable, get_id_field(OtherModule),
JoinTable, get_id_field(Obj), get_id(Obj),
WhereClause,
ExtraClause]).
This can be cleaned up a little (and your erlsql example too) by using
some local variables. %L says the arg is a list of parameters.
If we use a parse transform, the it may look even prettier (in my
eyes, at least)
make_get_related_many_to_many_query(OtherModule, JoinTable, Obj,
WhereClause, ExtraClause) ->
sql:build("select %%OtherModule:shown_fields() "
" from %%[OtherModule:table(), JoinTable] "
" where
(%OtherModule.id=%JoinTable.%get_id_field(OtherModule) "
" and %JoinTable.%get_id_field(Obj)=%get_id(Obj)) "
" and (%WhereClause) "
" %ExtraClause").
What sql:build does behind the scenes is an implementation detail. It
may put together strings (like io:format) or it may parse the sql,
build a quasi-sql representation and work with it. As a user I don't
care, as long as it does what it is supposed to do. I don't have to
keep counting nested {}s and []s and to learn what in fact is a new
query language (you already want to introduce new ways to write
things).
Whether these examples look better or not than ErlSQL is in the eyes
of the beholder. I, personally, don't find ErlSQL expressions any
worse than the alternatives, but I would be happy to give the user
both options and let them pick whichever they prefer :)

To ErlSQL's advantage, I should say that emacs does a good job of
indenting your clauses and helping you balance all the parens, and it
has nice syntax colorization that make these expressions look better
than they do in Email. Also,
it's much cheaper to build ErlSQL in runtime than the first example because
it doesn't require any parsing.

Another benefit of ErlSQL is that when you pass an ErlSQL expression
as a parameter to different functions, it's easy for them to extract
information about its structure and values using pattern matching. In
ErlyDB, this makes writing adapters for non-SQL drivers (e.g. Mnesia)
much easier.
Post by unknown
BTW, how do you write 'join' statements? The natural way for an erlang
data structure would be {join, {select,...}, {select,...}}, but that
doesn't follow the way the sql looks like.
Right now explicit joins aren't part of the language simply because I
haven't needed them. Joins are implicitly expressed in FROM and WHERE
clauses.
Post by unknown
Having a new query language isn't a bad thing, but then it's not sql.
It's not SQL... it's an Erlangification of SQL :)

Cheers,
Yariv
unknown
2006-09-21 10:29:03 UTC
Permalink
Post by unknown
How could the Erlang compiler help me write better SQL
statements? Yes, if I write quasi-SQL, I get some syntax
checking, but I still can write stupid things like
{select, {where, {x, '=', 3}}, foo, {from, [table]}}
which is a correct Erlang term, but not correct quasi-SQL.
Well, that would be the task of a domain-specific parser,
or a parse_transform.

Basically:

- pure SQL in a string or binary
+ nothing can be decided at compile time, except perhaps
that it is a string or a binary.

- a record or list of atoms etc. mimicking SQL
+ syntactic checks can be made at compile time, but no
semantic checks, like the one you describe

- a very regular structure with a library doing lots of
pattern matching
+ dialyzer can possibly, at "compile time", determine
that your statement will not pass the pattern matching
of the library, since it's not properly structured.

- a domain-specific parser
(either as ROK suggests, or perhaps by wrapping the
query inside a pseudo function and checking it in a
parse transform, much like with QLC and Mnemosyne)
+ extensive checks and optimizations can be made at
compile-time.

With the last option, one has to decide whether to stick
to regular Erlang syntax, or depart from it. It doesn't
preclude using plain SQL syntax, since you can call an
SQL parser from the parse transform. If you want a yecc
grammar for SQL, there is one, actually, which works
at least for a significant subset of ANSI92 SQL.

Departing from Erlang syntax is tricky in a parse transform,
since the erlang tokenizer and parser will get hold of it
before your parse transform. Using erlang syntax, but
imposing your own semantics is likely to confuse the
reader. I did that to some extent in plain_fsm, by wrapping
a receive statement inside a pseudo function in order
to handle OTP system messages. The obvious dilemma was
that pattern matching can't be extended that way, so
the parse transform had to turn it into something quite
different. This has to be clearly explained, and the
sought advantage is partly spoiled.

What would be very nice, I think, would be to write a
QLC generator for e.g. MySQL. Nice - if someone else
decides to do it, and I wouldn't have to. (:

A MSc student did write an SQL parser converting SQL
to Mnemosyne queries. This worked quite well. Going
from a QLC query to SQL should be doable, and - assuming
I'm right - raises the argument that Erlang already
_has_ quite a powerful syntax for database queries.
So why invent a new one?

BR,
Ulf W
unknown
2006-09-21 10:38:09 UTC
Permalink
Post by unknown
Post by unknown
How could the Erlang compiler help me write better SQL
statements? Yes, if I write quasi-SQL, I get some syntax
checking, but I still can write stupid things like
{select, {where, {x, '=', 3}}, foo, {from, [table]}}
which is a correct Erlang term, but not correct quasi-SQL.
Well, that would be the task of a domain-specific parser,
or a parse_transform.
Yes, of course. This is what I was trying to say too, but didn't
manage to express my self as clearly.
Post by unknown
Departing from Erlang syntax is tricky in a parse transform,
since the erlang tokenizer and parser will get hold of it
before your parse transform.
This might get a solution, "Quite Soon (tm)"-ish.
Post by unknown
A MSc student did write an SQL parser converting SQL
to Mnemosyne queries. This worked quite well. Going
from a QLC query to SQL should be doable, and - assuming
I'm right - raises the argument that Erlang already
_has_ quite a powerful syntax for database queries.
So why invent a new one?
Very good point. But this wouldn't make the discussion obsolete
because there might be other DSLs that one might want to use besides
sql.

best regards,
Vlad
unknown
2006-09-22 00:36:57 UTC
Permalink
"Ulf Wiger \(TN/EAB\)" <ulf.wiger> quoted:
Do you mean teaching the compiler to parse SQL, or to leave
it as is and letting the compiler remain ignorant? ;-)

The approach I suggested, of having a declaration to associate
an "embedded syntax" keyword with a new parser module, is, really,
BOTH:
- the Erlang compiler only knows about Erlang
- but it can be extended with anything that can turn a
token list into Erlang abstract syntax trees

It's a parse transform at a somewhat lower level.
unknown
2006-09-22 06:56:03 UTC
Permalink
Post by unknown
Do you mean teaching the compiler to parse SQL, or to leave
it as is and letting the compiler remain ignorant? ;-)
The approach I suggested, of having a declaration to associate
an "embedded syntax" keyword with a new parser module, is, really,
- the Erlang compiler only knows about Erlang
- but it can be extended with anything that can turn a
token list into Erlang abstract syntax trees
It's a parse transform at a somewhat lower level.
Yes, and I started writing a reply to your post
telling how much I liked your idea. I think it
should be quite easy to implement, too.

I've forgotten why I didn't send it. (:

BR,
Ulf W
--
Ulf Wiger
unknown
2006-09-22 07:11:38 UTC
Permalink
Post by unknown
Post by unknown
The approach I suggested, of having a declaration to associate
an "embedded syntax" keyword with a new parser module, is, really,
- the Erlang compiler only knows about Erlang
- but it can be extended with anything that can turn a
token list into Erlang abstract syntax trees
It's a parse transform at a somewhat lower level.
Yes, and I started writing a reply to your post
telling how much I liked your idea. I think it
should be quite easy to implement, too.
It is easy. :-) The big problem is to decide on the syntax and then to
convince the OTP team that it's a good idea to get it into the
mainstream.

With a little cooperation from the respective parser, it is also
possible to use such expressions as patterns in a match. With
provisional syntax
<sql| select @Fields from users |> = <sql| select name, id from users |>,
Fields == [name, id]

I will publish[*] a document about my attempts at metaprogramming in
Erlang, where even more extensions are described. In the worst case,
it would be possible to have the extensions only present in
"metaerlang" files (suggested file extension: .mrl), so that we don't
pollute the regular erlang language.

[*] I will begin this weekend and have it on trapexit.

regards,
Vlad

Continue reading on narkive:
Loading...