28-Mar-2024 13:12 GMT.
UNDER CONSTRUCTION
Anonymous, there are 15 items in your selection
[Files] [MorphOS] SeventhSense updatedANN.lu
Posted on 12-Sep-2004 19:46 GMT by Grzegorz Kraszewski15 comments
View flat
View list
SeventhSense, the ultimate website scanner has been updated with new features like HTTP proxy support, configurable numbers of threads and more. Many bugs have been fixed. Download from MorphZone. From the change log:

- BUGFIX: Fixed keyboard handling of GUI (missing MUIA_CycleChain here and there). Bug reported by Christian Rosentreter.

- BUGFIX: Version string was a bit malformed. Bug reported by Christian Rosentreter.

- FEATURE: Sites list autobackup. When the default sites list is saved, an old one is renamed to "sites.bak" first (old "sites.bak" is deleted). Feature suggested by Christian Rosentreter.

- BUGFIX: When a list is loaded into program, entries are checked for duplicated IDs. If duplicate ID is detected, user can skip duplicate or change ID. Bug reported by Grzegorz Murdzek.

- BUGFIX: When iconified SeventhSense displays now its own icon instead of default one.

- FEATURE: "Delete" function moved from toolbar to context menu.

- FEATURE: Checking can be stopped for every site separately with "Stop" option in the context menu.

- FEATURE: All check processes can be stopped at once with "Stop all" button.

- BUGFIX: SeventhSense could try to load a site in an endless loop in case of 40x HTTP response (but other than 404).

- FEATURE: User configurable number of simultaneously checked webpages.

- FEATURE: WWW proxy support.

- FEATURE: Option to automatically open an updated website in a browser. Feature suggested by Grzegorz Murdzek.

- BUGFIX: If there is no year in the date pattern, current year is assumed. Bug reported by Grzegorz Murdzek.

[MorphOS] SeventhSense updated : Comment 1 of 15ANN.lu
Posted by catohagen on 12-Sep-2004 18:33 GMT
morphos freezes instantly when i try to check for updates on a site, what can i do wrong ?
[MorphOS] SeventhSense updated : Comment 2 of 15ANN.lu
Posted by Grzegorz Kraszewski on 13-Sep-2004 05:14 GMT
In reply to Comment 1 (catohagen):
Send me your sites.list via e-mail first. I will check and then we can take further steps to debug it.
[MorphOS] SeventhSense updated : Comment 3 of 15ANN.lu
Posted by Grzegorz Kraszewski on 13-Sep-2004 08:07 GMT
In reply to Comment 1 (catohagen):
Bug seem to be tracked, the solution is using "multimedia.class" from 0.12 release, or create an empty directory "MOSSYS:Classes/Multimedia". The third solution is waiting for the next update (it is planned on the next Sunday).
[MorphOS] SeventhSense updated : Comment 4 of 15ANN.lu
Posted by Marcus Sundman on 13-Sep-2004 16:52 GMT
Since SeventhSense (SS) is the "ultimate website scanner" it should be able to do at least what other "normal" website update checking programs can, right?
So, can SS display the difference between two versions of a web page?
Can SS store the changes locally so that the user can retrieve a set of pages from a particular point in time?
Can you specify exact patterns of what changes to look for and what changes to ignore?
Can SS track changes recursively (tracking new pages as they appear)?
[MorphOS] SeventhSense updated : Comment 5 of 15ANN.lu
Posted by pab on 14-Sep-2004 04:11 GMT
Fuck al mos users in the world.

aos 4 rules
[MorphOS] SeventhSense updated : Comment 6 of 15ANN.lu
Posted by hooligan/dcs on 14-Sep-2004 04:33 GMT
In reply to Comment 5 (pab):
Who is Al Mos?
[MorphOS] SeventhSense updated : Comment 7 of 15ANN.lu
Posted by stefkos on 14-Sep-2004 05:45 GMT
In reply to Comment 6 (hooligan/dcs):
He want to say KMOS.
[MorphOS] SeventhSense updated : Comment 8 of 15ANN.lu
Posted by Grzegorz Kraszewski on 14-Sep-2004 12:19 GMT
In reply to Comment 4 (Marcus Sundman):
Since SeventhSense (SS) is the "ultimate website scanner" it should be able to do at least what other "normal" website update checking programs can, right?

Right. What other "normal" website update checking programs do you mean? GuardAmy? Of course I mean programs running under MorphOS.

So, can SS display the difference between two versions of a web page?

Not yet.

Can SS store the changes locally so that the user can retrieve a set of pages from a particular point in time?

Yes. All downloaded pages are stored as disk files.

Can you specify exact patterns of what changes to look for and what changes to ignore?

SeventhSense is date-based. It scans webpage HTML code for dates. And of course date patterns are defined by user.

Can SS track changes recursively (tracking new pages as they appear)?

New pages as they appear where? SeventhSense is not designed to be a search robot like ones used by Google. It requires very high connection bandwidth to actively search the whole Internet.

Thank you for your ideas. They will be used to improve SeventhSense, and remember it is still beta (0.x).
[MorphOS] SeventhSense updated : Comment 9 of 15ANN.lu
Posted by Grzegorz Kraszewski on 14-Sep-2004 12:23 GMT
In reply to Comment 5 (pab):
Well, it is all you can do, useless trolling, mr AmigaOS4 fan. One of the best way to promote a platform is writing software for it. That is what I do for MorphOS. What do you do for OS4? It is clearly visible MorphOS wins the battle on software ground. And writing such useless, off-topic commenst won't help OS4 at all.
[MorphOS] SeventhSense updated : Comment 10 of 15ANN.lu
Posted by Marcus Sundman on 14-Sep-2004 17:28 GMT
In reply to Comment 8 (Grzegorz Kraszewski):
> > Since SeventhSense (SS) is the "ultimate website scanner" it should be able to
> > do at least what other "normal" website update checking programs can, right?
>
> Right. What other "normal" website update checking programs do you mean?
> GuardAmy? Of course I mean programs running under MorphOS.

OK, I didn't think "ultimate website scanner" meant "best website scanner available on MOS".

> > So, can SS display the difference between two versions of a web page?
>
> Not yet.

It should. Displaying what has been updated is one of the most basic features there is for programs tracking website updates.

> > Can SS store the changes locally so that the user can retrieve a set of pages
> > from a particular point in time?
>
> Yes. All downloaded pages are stored as disk files.

OK, good. Is there a way to limit this, so that the database won't just keep getting bigger and bigger?

> > Can you specify exact patterns of what changes to look for and what changes
> > to ignore?
>
> SeventhSense is date-based. It scans webpage HTML code for dates. And of course
> date patterns are defined by user.

I don't see any sense in that. Why look for dates? Why not look for changes and then simply store the date and time the changes were seen?

> > Can SS track changes recursively (tracking new pages as they appear)?
>
> New pages as they appear where?

On the website you want to track, of course.
Let's take an example. Let's say there is this nice news site that has a front page with links to the actual stories, each of which is on a separate page. I want to tell my website tracking program to track the first page recursively with a recursion depth of 1 and without leaving the site. Now the program should see the new articles as they appear and store them etc. Also, if there are updates or corrections in older articles it should also see that.
OK, then let's say I change the recursion depth to 2. Now if articles refer to other pages on the same site the program will also start to track those. If I then specify that it may leave the site, too, then it will also track off-site pages linked to from the news articles.

Just make sure it's impossible to select both "may leave the site" and "infinite recursion depth", or else the program might start to download a very big part of the internet. :-)
[MorphOS] SeventhSense updated : Comment 11 of 15ANN.lu
Posted by Grzegorz Kraszewski on 14-Sep-2004 19:05 GMT
In reply to Comment 10 (Marcus Sundman):
OK, I didn't think "ultimate website scanner" meant "best website scanner available on MOS".

Well, ANN is concentrated on Amiga and Amiga-like operating systems. I think SeventhSense is the best scanner for these.

It should. Displaying what has been updated is one of the most basic features there is for programs tracking website updates.

SeventhSense is mainly designed for checking portals for news items. That is why scanning dates and odrinal numbers [of news items] is more important to me than diffing the whole HTML code. Also that is why I plan to add a simple RSS reader. Of course diffing will be implemented some day, but is not on top of todo list.

OK, good. Is there a way to limit this, so that the database won't just keep getting bigger and bigger?

Hard disks are big nowadays. SeventhSense stores HTML code only, so I don't think it is a problem. But if users demand the feature, why not. BTW you are using (or have tested at least) SeventhSense?

I don't see any sense in that. Why look for dates? Why not look for changes and then simply store the date and time the changes were seen?

Well, just diffing causes many "false alarms", when a webmaster, or editor just adds missing comma, or fixes page layout. And no, adding difference threshold does not help much. I used GuardAmy (it does diffing) and was disappointed with this. That is why SeventhSense bases on date scanning (it also checks for "Last-Modified" field in HTTP header if present, can be disabled for some sites returning date of request there).

Let's take an example. Let's say there is this nice news site that has a front page with links to the actual stories,

Then these stories have dates with them, or at least ordinal numbers in links (most of news sites use scripting and database). And if such a site has neither dates nor ordinal numbers, it is probably hand-edited and header date can be used. Automatic adding of sub-pages makes little sense here, as the database gets populated with useless rubbish, checking it is just wasting bandwidth (some people, like me for example, still have to use analog modem dial-up). It is very unlikely I implement such a feature in SeventhSense. If you want to mirror a website just use wget.
[MorphOS] SeventhSense updated : Comment 12 of 15ANN.lu
Posted by Marcus Sundman on 14-Sep-2004 22:43 GMT
In reply to Comment 11 (Grzegorz Kraszewski):
> SeventhSense stores HTML code only

Ouch! That can be very, very bad in certain situations. E.g. one might want to track some weather page which happens to display something with an image.

There are a lot of people who want to track a lot of different things for a lot of different reasons. It doesn't take very much additional effort to make a website tracker flexible enough for the majority of them, instead of just for very few specific cases. However, it's your program, you may do whatever you want with it. I just think it'd be nice to have a good website tracker for MOS and/or AOS.

> BTW you are using (or have tested at least) SeventhSense?

Nope. I just happen to be in the same business (although not for MOS/AOS).
I did download it and read the docs before my first comment, though. I also tried to look for the website, but I couldn't find it.

> > Why look for dates?
>
> Well, just diffing causes many "false alarms", when a webmaster, or editor just
> adds missing comma, or fixes page layout.

The wheel is spinning but the hamster is dead.
If there is a page that uses some "last updated" date/time then of course I can diff only that. There is no reason why I would have to diff the whole page just because I can. (Perhaps you missed the "specify exact patterns" part of my original comment.)

> > Let's take an example. Let's say there is this nice news site that has a front
> > page with links to the actual stories,
>
> Then these stories have dates with them, or at least ordinal numbers in links
> [...]
> some people, like me for example, still have to use analog modem dial-up

Umm.. wouldn't you want your program to at least download/cache those new news articles for you? Especially when you're on a dial-up, so that you can read them offline. You also want to see which news articles are new, right? It'd not enough to know that the main page has changed. It might have links to 100 news articles in different sections, so you really want to know which ones are new.

> Automatic adding of sub-pages makes little sense here, as the database gets
> populated with useless rubbish, checking it is just wasting bandwidth

What is useless rubbish to you might be very important to other people.
Also, you could use an exponential check delay. (E.g., first it checks every hour, after a while it's every 10 hours, then every 2 days, then once a week, and pretty soon it will reach whatever max delay you've specified. If there is a change it would start this from the beginning for the changed page.)

> If you want to mirror a website just use wget.

Why on earth do you think I might want to mirror a website?!? Isn't this about tracking changes to web pages? That's what I want to do. The program can't do that if it doesn't store the web pages. (If the program won't show the difference between the old and new version then it won't even have to store the web pages. It could just create an md5/sha/whatever for the parts matching the pattern you've specified, although then you'd have to re-download the pages whenever you change the match pattern.)
[MorphOS] SeventhSense updated : Comment 13 of 15ANN.lu
Posted by Grzegorz Kraszewski on 15-Sep-2004 06:48 GMT
In reply to Comment 12 (Marcus Sundman):
Ouch! That can be very, very bad in certain situations. E.g. one might want to track some weather page which happens to display something with an image.

Images are not downloaded. Period. SeventhSense task is just detect page update. Then I can doubleclick on and entry and it will be loaded in the system default web browser. Also note it is (yet) impossible to have embedded HTML viewer in MorphOS app (there is a class but it implements incomplete HTML 3.0 so it doesn't count). So how am I expected to display the diffs? As plain text? Text with HTML tags stripped? It can be unreadable for many pages.

There are a lot of people who want to track a lot of different things for a lot of different reasons. It doesn't take very much additional effort to make a website tracker flexible enough for the majority of them, instead of just for very few specific cases.

Well, the most flexible way to detect what has changed actually is just look at the page. SeventhSense is just a helper, it says "well from all those 126 websites 11 has been updated so you should take a look at them". Using too much automation can lead to ommiting some updates.

However, it's your program, you may do whatever you want with it. I just think it'd be nice to have a good website tracker for MOS and/or AOS.

Well we just have different definition of "website scanning". SeventhSense is designed to tell the user which sites of her list are updated and should be viewed with a browser.

Nope. I just happen to be in the same business (although not for MOS/AOS).

And what is the program name?

Especially when you're on a dial-up, so that you can read them offline. You also want to see which news articles are new, right? It'd not enough to know that the main page has changed. It might have links to 100 news articles in different sections, so you really want to know which ones are new.

The topmost ones... As I see you concentrate on scanning "wikipedia-like" sites, where updates may be made at any place and at any time. SeventhSense is geared for news portals, where items are chronologically sorted. It can't handle all sites, you can always design one for which SeventhSense will be useless. Probably uisng of word "ultimate" was the reason which triggered all the discussion, I'll be more careful with this word in the future.
[MorphOS] SeventhSense updated : Comment 14 of 15ANN.lu
Posted by Marcus Sundman on 15-Sep-2004 09:13 GMT
In reply to Comment 13 (Grzegorz Kraszewski):
> Also note it is (yet) impossible to have embedded HTML viewer in MorphOS app
> [...] So how am I expected to display the diffs?

The easiest way is probably to create a copy of the webpage, but with the changed sections highlighted, and then open that page in a web browser.

> > Nope. I just happen to be in the same business (although not for MOS/AOS).
>
> And what is the program name?

Actually we have several, but I'd rather not tell you the company name since I'm not sure about the legal consequences of that. (The last thing I'd want is that people would see me as representing the company in a forum like this.)

> It can't handle all sites, you can always design one for which SeventhSense
> will be useless.

Sure. I just think a little more effort would make SS suitable for many, many more scenarios. My opinion doesn't matter, though.

> Probably uisng of word "ultimate" was the reason which triggered all the discussion

Indeed it was.
(Still, I do think it'd be nice to have the ultimate website tracker available for MOS/AOS, just as we once had the ultimate IRC client available for AOS.)
[MorphOS] SeventhSense updated : Comment 15 of 15ANN.lu
Posted by Grzegorz Kraszewski on 15-Sep-2004 09:40 GMT
In reply to Comment 14 (Marcus Sundman):
Sure. I just think a little more effort would make SS suitable for many, many more scenarios. My opinion doesn't matter, though.

Well, all opinions matter to me. I may disagree with some, but take all into account.

(Still, I do think it'd be nice to have the ultimate website tracker available for MOS/AOS, just as we once had the ultimate IRC client available for AOS.)

Maybe SeventhSense will become at least "almost ultimate" some day. Anyway the next update is scheduled on September, 19th :-)
Anonymous, there are 15 items in your selection
Back to Top