This page will have all relevant email transactions with the students regarding the AI course, so that everyone has equal information regarding the course material.
Newest transactions will be posted on the top. Regularly view this page to see what's going on between other students and the instructor.
All sensitive material such as your name, email address, etc. will be removed, and also any part of code that is not relevant to the discussion will not be uploaded here.
****************************************************************************
Sat Dec 14 13:26:16 CST 2002
[Q] For the backprop timing vs. number of epoch experiment, do I repeat 5.1 or
5.2?
[A] Do it for 5.2, the hidden unit experiments.
****************************************************************************
Sat Dec 14 11:51:15 CST 2002
[Q] I was working on the perceptron part of the project, part 5.1 question
3(d). I looked at your table 1 and noticed that you have:
X Y | !(X /\ Y)
--------------
0 0 | 1
0 1 | 0
1 0 | 0
1 1 | 0
shouldn't this be:
X Y | !(X /\ Y)
--------------
0 0 | 1
0 1 | 1
1 0 | 1
1 1 | 0
Am I overlooking something or is the table wrong?
[A] You're right. That was another typo. Fix X /\ Y to X \/ Y.
****************************************************************************
Sat Dec 14 09:50:49 CST 2002
----------------------------------------------------------------------------
[Q] I am confused about the final exam material. Are you going to
put up a past exam?
[A] No. I will not put up a past exam. The exam style will be similar to
the midterm and you can look at the homework for a clue too.
As for the material, I won't ask you to derive the backprop learning
rule or anything. Just think about these things:
1. what is the core idea?
2. what are the major breakthroughs made by the algorithm? i.e.,
what kind of unsolvable problem in a previous
approach was solved in this one?
3. what are the major advantages?
4. what are the limitations?
That was for neural networks stuff.
As for other topics, think about the core concept. For example,
what is the single most important concept that allows Bayesian
updating to be efficient?
Think and study in that level.
There will be some problems as in the homework that may require
some precise application of your knowledge as well, so work on
that aspect as well.
----------------------------------------------------------------------------
[Q] What should the range of the weights be initially?
[A] Between 0.0 and 1.0, inclusive.
----------------------------------------------------------------------------
[Q] Right below figure 1., it says "Your program should take 5 inputs"...
Isn't it three?
max epochs
learning rate
function name to learn
[A] Yes, you're right. That was a typo.
----------------------------------------------------------------------------
[Q] I don't understand how to take in the input. Is is supposed to be
command line arguments? If not what?
[A] Just read a single line from standard input (scanf(...) or cin << ..., etc.)
----------------------------------------------------------------------------
[Q] Can I use GUI?
[A] I recommend that you do not use GUI at all, but if that's all you can
do (i.e., you cannot possibly do command line), then you may.
----------------------------------------------------------------------------
[Q] Some output can get really long -- for example, the weight dump
in the perceptron. Do you really want all that?
[A] If it is too long, you can just put the first 10 and last 10 lines
in the README file.
----------------------------------------------------------------------------
[Q] The sum of square error is the same. It's too long. What do I do?
[A] If you look carefully, the error stays the same for a long stretch
of time. Just report the epoch when it actually changes.
epoch error
100 2
..
1000 2
...
1989 2
1999 1 <---
...
----------------------------------------------------------------------------
[Q] For the backprop, how do I plot the error?
[A] First of all, the instruction in the README file is incorrect.
This will give you the epoch and error:
./bp conf/xor.conf | grep ERR
ERR 1 0.1xxxx
ERR 2 0.09xxx
...
However, there's that string 'ERR' in the front, so you don't
want that. To get rid of that, use awk:
./bp conf/xor.conf | grep ERR | awk '{print $2,$3}'
This will print the 2nd and 3rd ($2 and $3) column:
1 0.1xxxx
2 0.09xxx
...
Once you have this, you can plot it in any data or graphics
package. Excel may be useful if you don't know how to plot
in Matlab.
----------------------------------------------------------------------------
[Q] I cannot save the plot to an image file. What to do?
[A] Print it out and bring it to the final.
----------------------------------------------------------------------------
[Q] I am confused about how to plot the error curve. Do I plot all the
plots separately? or all in one? if not, how to group them?
This is about 5.2, experiment 1, and experiment 2
[A]
For experiment 1, each plot should be dedicated to one
function (one of AND, OR, or XOR).
1st plot: AND -- for alpha 0.01, 0.001, 0.0001
2nd plot: OR -- for alpha 0.01, 0.001, 0.0001
..
For experiment 2, the same goes:
1st plot: AND -- for 1, 2, 3, and 4 hidden units
2nd plot: OR -- for 1, 2, 3, and 4 hidden units
..
----------------------------------------------------------------------------
[Q] I am confused about the loop stopping criteria. If I cancluate the
squared error for each input pattern (for the 4 input patterns),
it would be different and I can get kicked out of the loop
as soon as at least one of the inputs give zero error. What do I do?
[A] That's why you want to use the "sum" of squared errors.
Sum ( Ti - Oi)^2
i = Input1 to 4
----------------------------------------------------------------------------
[Q] I am confused about section 5.1, 3(b). I don't know what you want?
[A] Think about the two problems in geometric terms. You may also run
experiments and find the average number of epochs to get an
empirical answer, from which you can think about the reason why.
----------------------------------------------------------------------------
[Q] In section 5.1, 3(c), ohw many times is "many"?
[A] Try 5 times each for the 4 inputs, and see if you get any meaningful
result. If not, try 10 times each, etc.
----------------------------------------------------------------------------
[Q] This is odd. When I run backprop with 1 hidden unit and bias turned
on, the output dump suggests that there are 2, not 1, hidden units.
[A] That is normal, because a bias unit is added at each input and hidden
layers, and is counted as an extra hidden unit by the program.
Don't be concerned about it and just set 1, 2, 3, and 4 for
the hidden unit size to do your experiment 5.2.
----------------------------------------------------------------------------
[Q] I programmed in JAVA, and it doesn't produce any executable. What
do I submit?
[A] Just submit the class file along with the other stuff.
----------------------------------------------------------------------------
[Q] Wait! Some answeres you gave above conflict with the answer you
gave me earlier!
[A] In that case, clearly mark that fact on your submission, and
you do not have to follow the guideline written above.
****************************************************************************
Wed Nov 13 11:40:15 CST 2002
[Q] I have a lot of questions about the written homework. Is there any
tip?
[A] See the graduate student board.
****************************************************************************
Thu Oct 17 12:06:11 CDT 2002
[Q] I can't figure out what that (funcall ...) in the (expand ...) function
in the skeleton code eight-util.lsp does!?
[A] You can safely ignore that function and write (expand ...) on your
own.
Here's how I was using it, but you don't have to do it this way at
all.
(defun dfs ...
...
(setq *expand-func* 'expand-uninformed)
...
)
(defun heuristic ...
...
(setq *expand-func* 'expand-informed)
...
)
(defun expand-uninformed (node)
...
; update depth d
...
)
(defun expand-informed (node)
...
; update depth d
; update heuristic h
...
)
So, the core routine did not have to worry about calling expand-uninformed
or expand-informed depending on the need.
****************************************************************************
Thu Oct 17 11:57:14 CDT 2002
[Q] My (append ....) does not seem to work in my (expand ...)!
[A] (append list-a list-b) RETURNS the appended list (list-a list-b).
It does NOT append list-b to list-a and update list-a.
So, to achieve that, you have to do
(setq list-a (append list-a list-b))
cf. list_a = append(list_a,list_b); /* in C/JAVA ..*/
For example, suppose you have a local variable that holds the
current list of nodes expanded from a certain node:
(setq tmp-expanded-node (cons (apply-op ...) tmp-expanded-node)
Note that here I used cons instead of append, because (apply-op ..)
returns a node, not a node list.
Here's an example:
>(setq x '(1 2))
(1 2)
>(setq x (cons 3 '(1 2))) ; notice that cons was used
(3 1 2)
>(setq x '(1 2))
(1 2)
>(setq x (append '(3) '(1 2))) ; here to use append, 3 had to be a list.
(3 1 2)
****************************************************************************
Thu Oct 17 11:54:46 CDT 2002
[Q] How do I make a state into a node? I have no idea.
[A] Node is a list of 4 items: state, heuristic value, depth value, and
a list of operators, which is the path.
In Lisp you use (list ...) to generate a list of objects:
(defun dfs (state)
...
(setq node (list state 0 0 NIL))
...
)
Basically, you make an initial node by making a list of the
provided state, heuristics 0, depth 0, and an empty list of operators
as the current path.
****************************************************************************
Thu Oct 17 11:51:58 CDT 2002
[Q] It is really hard to debug my code in Lisp. How do I get about locating
the exact cause of the error?
[A] Use the (trace ...) command to turn on the trace for the important
functions:
(trace dfs)
(trace dfs-core)
(trace expand)
(trace applicable)
(trace apply-op)
and then run your dfs
(dfs '(1 2 3 4 5 6 7 8 0))
The benefit of this is that it shows you what are the arguments
passed on to the functions and also what is returned.
You can then determine what was the flaw in your calling routine.
> (trace bfs)
(BFS)
> (trace bfs-core)
(BFS-CORE)
> (trace expand)
(EXPAND)
> (bfs '(1 2 3 4 7 5 6 0 8))
1> (BFS (1 2 3 4 7 5 6 0 8))
2> (BFS-CORE (((1 2 3 4 7 5 6 0 8) 0 0 NIL)))
3> (EXPAND ((1 2 3 4 7 5 6 0 8) 0 0 NIL)) ; expand is called
<3 (EXPAND (((1 2 3 4 7 5 6 8 0) 10000 1 (RIGHT)) ; expand returns this
((1 2 3 4 7 5 0 6 8) 10000 1 (LEFT)) list
((1 2 3 4 0 5 6 7 8) 10000 1 (UP))))
3> (BFS-CORE
(((1 2 3 4 7 5 6 8 0) 10000 1 (RIGHT))
((1 2 3 4 7 5 0 6 8) 10000 1 (LEFT))
((1 2 3 4 0 5 6 7 8) 10000 1 (UP))))
4> (EXPAND ((1 2 3 4 7 5 6 8 0) 10000 1 (RIGHT)))
<4 (EXPAND (((1 2 3 4 7 5 6 0 8) 10001 2 (LEFT RIGHT))
((1 2 3 4 7 0 6 8 5) 10001 2 (UP RIGHT))))
4> (BFS-CORE
(((1 2 3 4 7 5 0 6 8) 10000 1 (LEFT))
((1 2 3 4 0 5 6 7 8) 10000 1 (UP))
((1 2 3 4 7 5 6 0 8) 10001 2 (LEFT RIGHT))
((1 2 3 4 7 0 6 8 5) 10001 2 (UP RIGHT))))
5> (EXPAND ((1 2 3 4 7 5 0 6 8) 10000 1 (LEFT)))
<5 (EXPAND (((1 2 3 0 7 5 4 6 8) 10001 2 (UP LEFT))))
5> (BFS-CORE
(((1 2 3 4 0 5 6 7 8) 10000 1 (UP))
((1 2 3 4 7 5 6 0 8) 10001 2 (LEFT RIGHT))
((1 2 3 4 7 0 6 8 5) 10001 2 (UP RIGHT))
((1 2 3 0 7 5 4 6 8) 10001 2 (UP LEFT))))
If anything goes wrong in EXPAND, you'll notice it.
****************************************************************************
Wed Oct 16 17:05:36 CDT 2002
[Q] Why do I need to use (let ...) at all?
[A] Because if you have same variable names in two different
functions, they will interfere. Look at this example:
>(defun xx () ; define function xx
(setq x 10)) ; set variable x to 10
XX
>(defun yy () ; define function yy
x) ; return variable x
YY
>(xx) ; running xx will set x to 10
10
>(yy) ; even though x is inside yy, it will
10 ; be affected by the previous (xx) call.
****************************************************************************
Mon Oct 14 16:02:13 CDT 2002
[Q] Why doesn't my (let ...) statement work?
(let (current-node (car node-list)))
...
(setq node-list (append (expand current-node) (cdr node-list)))
...
[A] The (let ...) form is effective only within its parentheses. So, the
above example would be interpreted as the following, say in psudo-C:
{
TYPE current_node;
}
...
node_list = append(expand(current_node), cdr(node_list));
So, the assignment will fail.
The above should be changed to
(let ( (current-node (car node-list)) )
...
(setq node-list (append (expand current-node) (cdr node-list)))
...
)
Also note that the syntax for (let ...) is:
(let (var1 var2 var3 ...)
BODY
)
or, if you want to initialize (some of) the variables,
(let (var1 (var2 10) (var3 20) var4 ...)
BODY
)
Note that var1 and var4 are uninitialized, while var2 and var3 are
initialized to 10 and 20, respectively.
The best strategy is to have (let ...) to be the outer-most
expression in a function:
(defun fun-name (arg1 arg2 ...)
(let (var1 (var2 5) var3 ...)
BODY
)
)
****************************************************************************
Sun Oct 13 22:08:35 CDT 2002
[Q] My dupe check fails the second time I run any search function.
My explored node global variable is not empty when I start the
second call.
[A] You have to reset your explored node list before starting a new
search. Suppose *explored* was your list:
(defvar *explored* nil)
then within each search function, run
(setq *explored* nil)
****************************************************************************
Sun Oct 13 22:06:57 CDT 2002
[Q] Do I need to come up with a heuristic?
[A] No, just implement h1 (tiles out-of-place) and h2 (total manhattan
distance). If you want to try new ones, that's fine, but make sure
you do h1 and h2.
****************************************************************************
Sun Oct 13 22:05:55 CDT 2002
[Q] My dupe check fails! -- see below.
> (dupe '(1 2 3 8 4 0 7 6 5)
> '((1 2 3 8 4 0 7 6 5) (1 2 3 8 4 5 7 0 6) (1 2 0 8 4 3 7 6 5)
> (1 2 3 8 0 4 7 6 5) (1 2 3 8 4 5 7 6 0) (1 2 3 8 0 5 7 4 6)
> (1 2 3 8 4 5 0 7 6) (1 2 3 8 4 5 7 6 0) (1 0 2 8 4 3 7 6 5)
> (1 2 3 8 4 0 7 6 5) (1 2 3 8 4 0 7 6 5) (1 2 3 8 4 5 7 0 6)
> (1 2 0 8 4 3 7 6 5) (1 2 3 8 0 4 7 6 5) (1 2 3 8 4 5 7 6 0)
> (1 2 3 8 0 5 7 4 6) (1 2 3 8 4 5 0 7 6) (1 2 3 8 4 5 7 6 0)
> (1 0 2 8 4 3 7 6 5) (1 2 3 8 4 0 7 6 5) (1 2 3 8 4 0 7 6 5)
> (1 2 3 8 4 5 7 0 6) (1 2 0 8 4 3 7 6 5) (1 2 3 8 0 4 7 6 5)
> (1 2 3 8 4 5 7 6 0) (1 2 3 8 0 5 7 4 6) (1 2 3 8 4 5 0 7 6)
> (1 2 3 8 4 5 7 6 0) (1 0 2 8 4 3 7 6 5) (1 2 3 8 4 0 7 6 5)))
[A] Notice that dupe checks a "state" agains a "explored-node-list".
Note that this "explored-node-list" is different from the
node-list you have in the search function!
(dupe '(1 2 3 8 4 0 7 6 5)
(
((1 2 3 ..) 0 1 (UP)) ; node 1
((1 2 3 8 ..) 2 2 (DOWN LEFT)) ; node 2
...
)
)
****************************************************************************
Fri Oct 11 19:42:58 CDT 2002
[Q] I am not clear about how to check for duplicate states. Do I check
for duplicate against the node-list?
[A] No. The node-list only contains unexplored nodes that have been
expanded from other (already checked) nodes so far.
You may have to keep a global variable, say
(defvar *explored* NIL)
and whenever you pop one node to check for goal and expand, push
that node in the *explored* list. Keep updating this list whenever
you take a node out of the node-list, and when you generate new
nodes, simply run dupe to check if it occured in *explored*.
Updating and check for the *explored* list should be done
in (expand ...).
****************************************************************************
Fri Oct 11 19:41:08 CDT 2002
[Q] How do I write a loop in Lisp?
[A] In case you really want to write a loop, here's a simple example:
>(setq x 1)
1
>(loop
(setq x (+ x 1))
(if (> x 20) (return x))
)
21
In the "loop" function, when it encounters "return" it exits the
loop and returns the value specifiec by "return" as the value of the
whole (loop ... ) expression.
****************************************************************************
Thu Sep 19 13:52:20 CDT 2002
[Q]
> In the function deriv, how should we go about unary minus, since its symbol
> is exactly the same as the subtraction symbol. I understand how to struction
> the function derivminus, but am confused how to add its function call to
> deriv.
[A]
I discussed this in the class.
Basically, instead of checking if
(equal (first expr) '-)
add an additional condition
(and (equal (first expr) '-)
(equal (length expr) 3))
for binary, and check for length 2 for unary.
****************************************************************************
Thu Sep 19 12:34:35 CDT 2002
[Q]
> I noticed on the Novak code posted that the multiplication function doesn't
> simplify very much. For example, when I run (derivtimes '(* x x) 'x) it gives
> me (+ x x) as the answer (2x). In our code are we then supposed to call splus
> so that the answer simplifies down to 2x?
>
> I guess the basic nature of my question is this: is the Novak code including
> the multiplication as is and ready to go, or do we need to modify it further.
[A]
If your derivXXXX uses the simplification functions splus, smult,
etc. in place of (list '+ ...), (list '* ....) etc., then that's
fine and no further simplification is necessary beyond that.
****************************************************************************
Thu Sep 19 12:17:52 CDT 2002
[Q] My program returns an answer that is mathematically correct but it
is different from the test output you provided in the lecture slide.
Is this okay?
[A] Of course! As long as your result is mathematically correct, you're fine.
****************************************************************************
Thu Sep 19 09:59:18 CDT 2002
[Q] Do I need to use "tar" to create a single file when submitting?
[A] Well, if you find it difficult, you may just submit individual files:
turnin 420-502 deriv.lsp README another-files yet-another-file
After that, to check if everything was submitted correctly, do
turnin -c 420-502
****************************************************************************
Wed Sep 18 14:34:21 CDT 2002
[Q] How do I deal with unary minus?
[A] First, the derivation rule is:
d(-f) d(f)
----- = - ----
dx dx
Next, the problem is how to simplify it. When given
(- expr)
the simplest one is:
if (expr == 0), return 0.
If you want to go a bit further, you can reduce
(- 12)
to just
-12
So, basically generate a negative number.
You can do this by first checking if expr is a number, and returning
(* -1 expr)
****************************************************************************
Wed Sep 18 12:11:31 CDT 2002
[Q] What do I do if the denominator is zero in case of division?
[A] Just check for that condition and generate an error message and
terminate the program.
****************************************************************************
Wed Sep 18 11:59:51 CDT 2002
[Q] How do I save the output from the GCL runs?
[A] Use the unix command "script" to save your entire session to a text file.
Bold text is what I entered. Italisized text is the
command prompts. The "script" command allows you to save whatever
appears on the screen in to a file you designate (below, it is
outputfile). Beware!: script overwrites the file,
so be very careful when running the script command. If you did
script deriv.lsp, that would be a disaster!!.
unix:~/> script outputfile
Script started, file is outputfile
%gcl
GCL (GNU Common Lisp) Version(2.3) Wed Feb 14 16:09:29 CST 2001
Licensed under GNU Library General Public License
Contains Enhancements by W. Schelter
>(load "deriv.lsp")
Loading deriv.lsp
Finished loading deriv.lsp
T
>(deriv '(* x x) 'x)
Error: The function DERIVMULT is undefined.
Fast links are on: do (si::use-fast-links nil) for debugging
Error signalled by COND.
Broken at COND. Type :H for Help.
>>:q
Top level.
%exit
exit
Script done, file is outputfile
****************************************************************************
Wed Sep 18 11:52:04 CDT 2002
[Q] How do I save the programs I typed in inside a GCL shell?
[A] There's no way of saving your work typed in inside of GCL.
You should write a separate program file on the disk and run
(load "deriv.lsp")
inside GCL to test the code.
1. edit your program in deriv.lsp, for exmaple with pico.
2. start gcl
3. inside gcl, run
(load "deriv.lsp")
4. still inside gcl, run
(deriv '(* x x)' x)
and etc.
5. If there is an error, type
:q
and edit your file deriv.lsp. Reload deriv.lsp in gcl
(load "deriv.lsp")
and test it again.
(deriv '(* x x)' x)
*. So, basically, you need to have two terminal windows up, one
for editing the code, and one for testing the code (in gcl).
****************************************************************************
Wed Sep 18 11:50:08 CDT 2002
[Q] What do I need to put in the README file?
[A] Briefly describe how your program works, and show example runs.
Discuss what you think about the program: is it intelligent? Why so or
why not?
****************************************************************************
Wed Sep 18 11:50:01 CDT 2002
[Q] Can I use the deriv.lsp on Gordon Novak's ftp site?
[A] Yes, you may. I made a link to that code in the news box. It includes
a simple implementation of multiplication as well as the addition
routine.