Categorically, neither one. And AIS and Radar systems have their own merits and in a perfect world the inputs from the both systems will be combined and represented to the navigator on a single integrated display (it was demonstrated in land-based trials with a remarkably good correlation among the 2 systems).
AIS vs Radar
In the merchant market in the near future it is probably that AIS and Radar systems will be required, however in the free market, other rules apply. If the AIS units might be made very cheap enough, then they are probably to find a ready market, much more so than the more expensive radar unit and particularly as the installation of the AIS on a little ship is considerably easier than a radar.
AIS system is working best when the mass of ships have installed that kind of AIS system onboard. Though a considerable advantage to the little vessel, seafarer may still be attained with using a Class B unit just to monitor the activities of all Class A equipped in the area and hence, to keep out of their way. There is a secondary issue, the Class B transmitter is also going to make the smaller ship visible to the Class A ships, though the Class A ships still need to keep an adequate look out for those boats and ships not equipped with any transponder at all. This kind of AIS system also contributes for the AIS live vessel tracking to all ships fitted with AIS transponders.
On the other hand the Radar is a stand-alone unit which doesn't rely on any other ship to be equipped with anything else than a good reflector and so is goint to operate in isolation to any other system. AIS needs, which other ships also be fitted to be visible. However, in the free market, the final problem is probably to come down to the price. Small ships radar are likely nearing the end of their design cycles and due to the mechanical components required are not probably to drop considerably in the near future. Class B AIS transponder on the other hand is designed specifically to be low cost, and, particularly with the recent introduction of integrated circuits which perform many of the coding and formatting functions that are required, potentially might reach a similar cost to existing VHF / GMDSS transceivers that have enough aproduction volumes. That factor on its own could be the basic factor in taking decision in the success or failure of the AIS market and vessel tracking.
AIS - The Ambitious Safety System
In its conception AIS (Automatic Identification System) looked exactly like that - a tool for identification of visually observed vessels or ships on the radar.
With it watchkeeping officers can call by name or call sign signs observed vessels to clarify their intentions. Thus, AIS will contribute to the avoidance of collisions and increase safety.
For example, it turned out that in some cases - particularly for small vessels not covered by SOLAS, AIS can actually replace the radar, given its ability in a single system to notice the presence, location and identity of other ships. Furthermore, the AIS can "see" around corners and noses which prevent the return radar signal, and usually has a greater range without the complexities of the radar. Thousands of owners of recreational and other vessels already use AIS instead of the radar as a tool for safe navigation.