Having gathered all this information about other people' work, you need to analyse it, and apply it to our own problem and product. Somewhere along the line you will hopefully also bring in your own ideas.
However, your job does not stop there, you will also need to look back at your product critically. Part of your final report should detail where you feel the faults and limitations of your work lie and in what way the product could be improved.
Consider first the analysis stage. You could, of course, simply gather your background information, let it permeate into your brain and then start hacking. Whether or not you succeed, you won't get many marks! Instead you should aim for some sort of structured review of the existing work. This will enable you to relate it to your own product (in your report), but also should help you to develop a better product.
One way to analyse the existing work is to simply to classify it into areas. If you are developing a database interface for the motor industry, you may have studied some papers and books on HCI, some on databases, and some articles about cars. If you have used filing cards to record your references, you could actually sort them into piles, if you have used a database, you could enter this classification as an extra field and use it to sort your bibliography. However, unless you have vast numbers of references simple hand sorting will be fine.
A more sophisticated alternative is to build some sort of taxonomy, sub-dividing your classes into sub-classes, sub-sub-classes etc. For example, you might divide the database systems you have studied into object-oriented vs. relational.
Beware however, of classifications (like those above) which follow too closely the standard assignment into computer science courses. You need to classify in a way which is pertinent to your project. One way to look for alternative classifications is to start off with the 'obvious' one and then force yourself to re-sort using some other criteria. In general, the more perspectives you take on a problem the more you will understand it.
If you have classified using several criteria then one way to organise your material is in a matrix. Use one classification along the vertical axis and one on the horizontal axis and simply place systems and papers in the relevant cells. If a cell is empty this might suggest that this is a particularly hard combination, or alternatively that it is a possible area to generate some fresh work.
Alternatively, you may notice that most of the cells are empty. This might suggest that there is some connection between the different classification criteria. The reason for this might become obvious as soon as you think about it, or you may find that you have discovered a previously undiscovered law (well it's worth being optimistic).
For the purposes of developing a new product, the most important feature of such a classification scheme is that it gives you a way of looking at your context. Consider your own product and where it ought to fit in the matrix. Is there an existing system/paper there? If so then it is a potential solution. If not (which is normally the case!) then you have to think a little harder. It may be that it is a particularly hard or even impossible situation, in which case, you need to consider ways of simplifying or altering the problem to make it soluble. More often it is simply that no-one else has done exactly the same thing before. In this case you look to previous work which share most of the same features as your own problem. In the simple two classification matrix this would mean simply looking to those systems in the same row or column as your own. Bringing features of these related systems together is a good first step to solving your problem ... watching out for crocophants on the way!
So much for using existing work to drive your own. What about all those bright ideas of your own. Well the first rule is don't rely on them. In the first Pert chart below, the bright ideas are on the critical path. This is a risky strategy. If you don't get the ideas you can't make the product and you will fail your project! However many bright ideas you normally get, don't rely on them coming when you need them.
However, the other extreme is to take a risk-free strategy. This is usually a minimax plan, which minimises your maximum losses. Unfortunately minimax plans for losses tend to be also minimax for gains. If there is no room for your own ideas in your project plan, then you can get by, but are not going to get really good marks. The ideal plan looks like the second pert chart. A risk free strategy which is still open to using your ideas when they come. A tall order, but not impossible.
So, now the hard part, how do you come up with all these bright ideas to feed into your work. Of course, there are no easy answers otherwise they wouldn't be bright ideas. However, there are some heuristics which can help. One thing to do is to abstract your problem. Look at it and think of a more general problem of which it is an example. Once you have done this you can see other problems which are related to it. You can then take solutions to the related problem and apply them to your problem. This process of abstraction and analogy is typical of many academic disciplines, but can also be intensely practical.
As an example of this I recently published a paper which related a range of problems including burning the toast and waiting for email replies. The common theme was that there was some event (the toast being cooked, or the reply arriving) which happened after some delay from the thing you do to start the process (put the bread under the grill , or sending the original message). Seeing the common general problem enabled me to consider general solutions: aide memoir, electronic reminders etc. and then to apply the analysis to disparate case studies: air traffic control and car radios.
De Bono's lateral thinking is one of the accepted, but hard to implement, pieces of advice for thinking up novel ideas. The aim is to think in a way which is different from one's normal thought patterns. You can read De Bono's books for his advice on this, but note also that we have already discussed several ways to encourage this. For example, the search for different classification schemes was a way of forcing different perspectives. Also asking very deep 'why' questions is one way of pushing you away from the standard solutions.
Another way to encourage new ideas is to constantly challenge yourself. You have had one idea, OK so what's wrong with it. Even if you think it is a good idea take it apart and see what's good and bad about it. In doing this you will be forced to think again. Imagine you are going to be given £10 for each fault you find. Then argue back to yourself.
It is hard to be suitably critical with your own ideas because they are your ideas and you like them! You obviously need to believe in what you are doing, but the emotional commitment to your ideas can prevent you from seeing the problems and hence find better ones. One way to get round this is to think up silly ideas. The advantage of silly ideas is that you don't intend them to be good ideas and so can feel happy criticising them. However, you can also look for good points. Just as you did with other people's work, take the silly idea and pretend that it is serious. Why is it good, why is it bad? The very silliness of it can help to make new connections and hence inspire new ideas.
Imagine a world famous expert on computer networks (now in his late 70s) has come to give a lecture. He is the inventor of the Internet, the World Wide Web was his brainchild and it is widely rumoured that Novell employed him as chief consultant. Despite his age he is still enthusiastic in his presentations and producing exciting new work. he stands up to speak: "Ladies and gentlemen, the problem with computer networks today is that they are like spaghetti bolognese without the Parmesan". As he speaks his excitement rises and then with the word "Parmesan" he suddenly clutches his chest and expires. The hall is shocked and when the doctor arrives he is pronounced dead. However, after the shock of his death subsides the audience begins to ponder these final words of wisdom from the master.
Think about it yourself, he said like spaghetti bolognese, not lasagne mind you, so he was obviously not thinking about OSI layers! Think of your own problem - is it like spaghetti bolognese without the parmesan. Think of a reason why it is. Now think of our own silly idea. They do not have to be quite a silly as that. I once was worrying about the problem of type-ahead and how you could tell whether the computer still had keystrokes to process. I suggested (as a silly idea) that the characters could appear a the bottom of the screen as they were typed and then a munchman could gobble them up as the application processed them. A silly idea, but capturing some good points.
You now have had your ideas and produced your product. The job is not over. The time has come to evaluate your product. This is clearly important from the client's perspective - does it do its job? However, you can also consider the whole project as a bit like a white coated scientist's experiment. In doing the experiment (making the product) you have tested your ideas and will then know more about them. Of course, without effective evaluation you won't know how good you were this time and so won't be able to do better next time. Also unless you evaluate your product effectively you won't get good marks.
Depending on the kind of product, various forms of evaluation might be appropriate. You might use test data or possibly run some sort of simulation to test your system. You might even attempt some sort of proof that it works. This proof might consist of a careful argument, or might even be a formal mathematical proof. In some cases, some statistical experiments or tests might be used, possibly in conjunction with simulation data, or data collected about use. Finally, do not underestimate anecdotal evidence, the comments of your client on seeing the product, your own experiences etc. In many areas it is impossible to obtain reliable objective evidence and anecdotal evidence is likely to be more useful, especially when considering future action.
So, we have come full circle, from analysing other peoples work to producing your own. Now, you can use the same techniques to analyse your own work as you used with other people's work. However, you have the added job of presenting your work in the best light to your examiner. This is bit like a marketing exercise, but with some important differences.
First of all, consider the good points. Identify them and report them. Tell the reader of your report what is good and why. Your product may be excellent, but you ought to be able to say why and relate your success (or failure) to the theory you used. If the examiner has to work this out then he or she is doing your work and you will be marked accordingly. If you don't say what's good, how does the examiner know it is not simply a fluke?
Equally important is to analyse the bad points. As I've said before, this is an important part of your project report. If your product has faults which your examiner notices, but you have not mentioned, then you will lose marks. If, on the other hand, you report any weaknesses and analyse why they happened, how it could be dealt with and what you have learned from the experience, then you are likely to gain marks.
Assuming you have identified the bad points in your product (and also in your project management etc.) you will need to examine the reasons for them. More why questions. It may be that you were limited by the available resources: software or hardware. Of course, in a commercial project you would be resource limited, so this is no excuse for sloppy planning! Your most critical resource will be your time and it may be that within the time available you can only produce a reduced form of the product you would like to have developed. This is something you should be addressing during the first semester as you produce your project plan and specify the scope of your project. However, you will almost certainly find that things do not go entirely to plan.
It may be that you simply lacked experience in some area and so things went wrong. Hopefully, your supervisor will be able to spot some of these problems before they happen, or they may become apparent as part of the first semester assessment. However, the project is your own independent work and as such you will make some mistakes. Remember that the project is part of your degree course and is a learning experience, you are not expected to know everything before you begin. If something went wrong because of lack of experience and you have identified the problem - then that is evidence of learning and will be to your credit.
Finally, there may be factors completely beyond your control. You may choose to use some software package which purports to perform all the functions you require, but half way through your work you find it is not up to the job. Faced with such difficulties you are expected to cope as well as you can: find workarounds, perhaps modify your aims. The important thing is again that you identify these problems and describe your methods of dealing with them.
Alan Dix 12/8/97