13-Jun-2024 08:20 GMT.
[News] Frieden brothers talk about OS 4ANN.lu
Posted on 27-Nov-2001 15:40 GMT by Teemu I. Yliselä125 comments
View flat
View list
The Frieden brothers from Hyperion have been answering questions about AmigaOS 4 at comp.sys.amiga.games in the recent days. I've c&p'd their answers below. [Warning, it's rather long - Teemu]

Note: the comments by the Friedens are in normal text, and the comments by other people are in italic

First of all, the idea is to get the AmigaOS from 68k to the PPC. Of course, there's a lot of code in the system, and lot's of code that's beyond our reach (hardware drivers in ROMs, for example). So one of the core components is of course the 68k emulator. The JIT version is already very fast, if you have a 040/25 and a fast 603e, you'll actually be able to run some programs faster than on the physical 040/25...

The second core component will be Exec SG. This is a reimplementation of the original Exec, written in C and placed onto a hardware abstraction layer so that it ports easily from one hardware to the other (remember, we intent to support a lot of different hardware, some of these will not have the Amiga chipset anymore). New features (compared to the old Exec) will be virtual memory. Don't worry, you'll not have to select "Shutdown" from the "Start" menu, you can just switch off ;) We also want to introduce some weak form of memory protection, like protecting code from being overwritten.. Later version will also include resource tracking.

The third mayor (new) building block is the new-style library interface. The cool thing about this is that it enables old 68k libraries to act as new OS4 libraries. To OS4 applications, 68k libraries will look like PPC native libraries. OTOH, new PPC only libraries will look like old 68k libraries to emulated 68k programs. This is achieved by the new library loader: Whenever a library is loaded, the "missing" part of the interface will be generated on-the-fly (by creating so called "stub" functions). So, even though a library is not yet ported to PPC, it can be used as if it were a PPC library, and as soon as it's ported, the program using it will use PPC code, even without recompiling the program (sorry if that comes out a bit complicated....)

For the initial release, we want to have at least the new Exec SG (otherwise, the whole system wouldn't work, anyway), a PPC native DOS, the TCP stack, new file system, and some devices. If there's still time, we'll port as much as possible to native PPC.

Nice. How about my 68060/50-604e/233 combo... What kind of speeds can I expect from the emulated 060 versus my real 50 MHz 060?

That's hard to say. The JIT compiler is still in the works, and the speed you get actually greatly depends on the program you're running - we have some cases where the speed is on par with a 060/50, and sometimes only with a 040/30. In any case, there's still a lot of room for optimization. As with any sort of emulation, statistical results may help finding those instructions that are executed most frequently and can receive special optimizations.

Is this based on the AROS work?

No, the ExecSG will be a complete re-implementation. AROS' exec is basically a re-write of the old exec, but SG will get a lot of new features, new library concept, Hardware abstraction layer etc.

With the virtual memory, will it be intelligent enough to shut itself off? Nothing is worse then the Windows implementation that once it needs to come on it stays on for the rest of the session whether it needs to or not.

No, there will be no shutdown required. Swapping will go to a dedicated swap partition preferably, and this will be re-initialized when the system starts, since the memory image on the swap space will be invalid anyway. So you can basically switch off the machine even when it's writing - no filesystem will get corrupted since there is no file system involved.

How is virtual memory better then spooling from the hard drive? What are the advantages?

Spooling from Harddrive is almost always the slower solution. For one thing, files on the harddrive are almost never in any format that the program in question can use out-of-the-box. For example, sound files may be encoded/compressed, and reading them from disk means decoding/decompressing them. Virtual Memory just dumps the memory image to the harddrive and reads it back when needed, no need to convert anything.

Awesome! Any hints as to how you pull it off? I know you can't give specifics, but I'd love to know anything about how that will work.

The libraries have an old-style jump table as well as a new-style symbol table once they are in memory. Both parts can be recreated using the FD file (which must be on disk obviously for this to work). The PPC side of things will use name binding, you don't have something like a library-relative jump but rather a textual symbol store in the file that gets "patched" at load time. This has the additional advantage of eleminating one jump - normally you do a "jsr -$nn(a6)" into the jump table and then take the "jmp $xxxxxx" that is stored there (on WarpUp it's slightly different but principially the same). The new system looks for the symbols at load time, and patches the subroutine jump directly into the executable, meaning there is only one direct jump into the library, not two.

Also, will my old PowerUP ELF's work at all on this system (there are only like 2 things I even remotely care about that aren't likely to see a Warp OS release)> This solution is Warp OS compatible, yes?

WarpOS uses a mechanism very similar to the 68k libraries. The basically work without system intervention, i.e. they use the jump table directly. Since the jump table is there, both 68k and WarpUp will work right away.

For PowerUP, I have to admit that I don't know how they work, but I am sure that it would be no problem to incorporate them. We're thinking of some sort of plugin interface for the loader, where you could write a small piece of code that can load a certain file type. This would for example make it possible to load Java JAR files, shell scripts, or any binary format transparently.

I have a Cyberstorm PPC, all my media (2 HD's and CD-ROM) barring my floppy are currently connected to the SCSI controller on my Cyberstorm. The HD's are formatted w/ the Trackdisk 64 filesystem provided on AOS 3.5/3.9... Are these still going to work w/ the new filesystem? If not, how can I migrate my data w/o "backing it up" elsewhere (eg: burning it to a CD)?

I think that the new file system is completely backward compatible, but I might be mistaken there. Olaf is doing it. If it doesn't work, we might provide a convertion program, but I am quite sure that the system *is* compatible.

What TCP stack are you going to use? I curretly use Miami DX, how does the AOS 4.x stack compare?

The TCP stack is the fastest one around, beating every other stack in terms of transfer rates. It gets over 900 kb/s on a Ariadne II board.

WHY OH WHY can't there be a Kickstart ROM upgrade?!? I'm sick of patches and kludges that reboot my machine before it's even up and working and require SetPatch or a mount list to mount my TD64 devices. Is there anything to be done?

Yes, a new kickstart would be good. We're thinking about that. I would personally prefer the kickstart to be loaded from disk (I mean, how doesn't have a harddisk these days?) and the ROM only containing a minimal bootstrap and drivers for booting the system (harddisk, minimal video driver for text messages, ethernet for BOOTP boots, CDFS for CD-ROM boots) and the rest being loaded from a harddrive.

With the multi-platform target, is optimization going to be possible, or will this be a straight compile of C source code?

There will be optimizations as far as possible, but most of the stuff will be plain C. The HAL will be the most critical part for a port.

Note, though, that although the prerequisites are there, a port to e.g x86 (I hear that coming :) is currently not planned, as our licence only covers PPC.

I see. How is this system going to integrate w/ the VP? Will there be two types of libraries? Executables?

Doing cross-calls to VP stuff is impractical, I don't think that there is such a mechanism. The DE will be integrated to a certain amount, but it will more or less stay a blackbox, a machine in the machine so to speak.

I like screens (dragable ones are nice but not required), I can't live w/o a dynamic RAM drive, I like push/pull gadgets, etc. Cosmetically Intuition/ GADTools need a serious overhaul, but the function is pretty much exactly the way I think a GUI should be. How will this compare in AOS 4.0?

The concept of screens is still in Intuition and will stick there. We are planning a major overhaul of Intuition, but that will have to wait until after the release. More modern GUI classes are required, and a real overhaul in favour of a skinnable, probably scaleable user interface.

Virtual memory is not really that much magic as it might seem. The new memory system will no longer be allocating every little block from the main memory list as the old one, it will work on a page basis. This will also prevent fragmentation: Multiple small memory allocations will be collected in pages (much like the memory pool system now). If it runs out of pages, pages in memory that haven't been used in a while will be swapped out.

Of course, when there's no room on the swap disk anymore, the system runs out of memory, and the allocation will fail. You can imagine that it's a no-brainer just to prevent the swapping: If the system runs out of physical pages, and swapping is turned off, it reports an error just as if the swap space is full...

The only problem with completely shutting down the VM system is if a large task has already swapped-out pages. In this case, turning off the VM will require the large task to page in it's VM stuff again, and this might not fit (remember that with VM, you can run tasks that are a multiple of the normal memory in your system). Of course, you can just quit the task...

It should also be said that the allocation of memory is under full control of the application. There will be an additional memory flag that will prevent memory allocations from being virtual.

But think about ROM based library (even if they are on disk), ie. the ones which are already in memory before any normal app can be started. SnoopDOS for example patches a lot of dos.library functions. So will you add lots and lots of callback support to dos.library functions, to make tools like SnoopDOS work?

SetFunction *only* works with libraries that are already loaded, so it's no problem to SetFunction anything on dos.library, or even exec.library. It would just change the value of the function's symbol in the library base, replacing it with another address. It's programs that are already loaded that will not use the new functionality. OTOH, chaning the workings of a function during program execution is very dangerous, anyway, so that could even be regarded as an advantage.

Fact is that SetFunction is a very dangerous functionality that is an invitation for every virus or misbehaving hack to make your system instable, unsecure, or crash. It has caused major headaches in the past, and should definitely be limited. The system will offer certain callbacks, and the rest is off limits. This will make the whole system a lot more stable.

A solution we're thinking about is loading a kernel image from the hard disk's boot block. The Amiga's disk structure allows arbitraty sized boot blocks, so theoretically, a whole kernel could fit there.

We will in any case offer the possibility to use something like SetPatch (including the dreaded reboot). This means that you do not have to reformat your harddrive or even repartition or something like that. As another possibility, we will try to offer the bootblock loading. We'll see if that works out.

There might be other solutions, but a update of the physical ROM is currently out of question. I wouldn't go for such a solution anymore, anyway, and rather prefer the mainboard to receive some sort of flashrom. We'll see what will be possible...

Once AmigaOS frees itself from original Amiga hardware, will we also lose compability games-wise?

We've contacted the author of WHDLoad, and we'll see what comes out of it.

We've also thought about the following: We discussed the possibility to include a UAE in the system (this is for later, mind you ;) that will kick in when binaries are executed that require the chipset. That way, running old games could be made transparent...

The other potential problem is that some programs did bad things with Exec. For example, some programs directly grope into the Exec memory lists. This is of course bad programming, but some games do that, and this will fail in OS4 because the memory subsystem will be completely replaced by the new VM stuff. We're keeping a small (resizable) memory pool for old stuff (for example, chip ram can't be virtual, so it's probably still there in it's old form).

However, what I fear the most are programs that assume, for example, stack frames or similar stuff. Chances are these will fail....

People wanting to develop for OS4 should for starts read the stuff included on all Developer CD's, a file named FutureOS or similar. It's very valid, and gives a lot of DON'Ts for future compatibility...

Honestly, OS4 will still be AmigaOS. That means that it will still be very small and efficient. The fact that we will have virtual memory will *not* turn it into Windows. You will still not need 70 MB just to boot, as it's in Windows 98...

The memory consumption will rise a bit compared to OS3.1 (mainly because PPC code is larger than 68k code), but the speed will rise as well... remember we have a PowerPC, which is a very powerful CPU, and this time, we can run it without the ball-and-chain 68k...

Most programs will run, however, those that for example - jump directly into the ROM
- mess around with ExecBase
- mess around with Intuition
- etc.

will have a hard time. I'm afraid that there will be some programs that will cease to work, however, this is inevitable, otherwise we would be dragging around a lot of old stuff for eternity.

Do you have any kind of ball park figure (percentage) of games that would fail with the 68 emulator (Exec?) ?

No, the emulator is still eveloving. Plus, we will have something in there that I will not talk about right now. Only so much: We will have facilities that will ensure maximum compatibility with maximum performance. I don't think more than 1% of the old code will fail...

Eyetech's AmigaOne has a FlashROM - could the whole Kickstart ROM fit in there (I don't know the size of it), rather than using it as a minimal boot ROM which then loads the proper "ROM" from HD?

Sure, we will use the AmigaOne's flash rom for booting.

a second question: can you tell us somthing about a maybe-cooperation with bplan?

We've talked to them.

If so, - do you get as one of the first a bplan-developer-machine?

We will apply for their developer program, but we will not get special treatment.

- are the drivers for various hardware motherboard-independent? (can i use GFX-cards, that don't would supportet in MorphOS but in OS4 (Radeon?)

That's a question I can't answer yet. The first version to come out will probably still have some chipset dependencies, that will be thrown out in various updates. So there's still some time until any other non-Amiga hardware is supported, anyway...

- will be the whole hardware supportet, f.e. FireWire drivers in OS 4?

See above. This will be decided as soon as we're actually running on the board. Support for FireWire will probably be in there at some point, but right now, it's not a priority.

- do i need different patitions for MorhOS and OS4

The OS4 library files will look somewhat different from the actual Amiga libraries (they are a different file format), so this might get you into trouble when MorphOS would try to open a new-style library. Theoretically, it would be possible...

Therefore most work is useless when noone develops with it, would it be possible to put a OS4.0 RKM - Amigaguide-file with all new OS4.0 related features documented on the CD too ?

We will make as much information as possible available as soon as possible. This includes documents as well as pre-release version of OS 4 for developers. Of course we want as much stuff on OS 4 as possible, so this is going to be a high priority.

Documentation will be available for download probably. I don't know if we do a CD, probably later after the release.

1. How much and what work was done on OS4.0 before it was handed over to Hyperion/Eyetech?

Mostly "secondary" technology, stuff like additional programs like the TCP stack and file system. We have some technology from H&P (like WarpUp, from which we will borrow code, and the H&P emulator). In fact, the "only" thing we are doing is Exec right now.

2. How integrated will the TCP/IP stack be? I'd rather have a real bsdsocket.library on disk and a separate config program than be forced to manually load it first.

I must admit that I don't know. I haven't seen it yet :/ I guess it will be an application program which will be launched from wbstartup...

3. Is sound support still targetted for OS4.2, or is this now coming in OS4.0?

We will have some new sound support already in OS4, but I don't know how much it will really be. As I said, we're doing the Exec, the rest will be done by others.

2) How can you possibly expect to get this done by February? ;)

Although Hyperion is doing most of the core work, a lot of work is carried out by others. Last time I counted there where a total of 18 people working on it; most of the developers from OS 3.5 and 3.9 are on board, as well as a few other very competent folks :-)

Back to Top