Talk:GrubWorkUnit
[edit] unsuccessful workunit upload (http put of arc.gz file)
hello there, should the .arc file (here on win32, with grubng c# on win32 version 01.2950.37591) and the arc.gz file actually vanish after a workunit is done and uploaded? my current grubng installation with the above version doesnt seem to succeed in uploading the workunit properly at the end, and these two files seemt to grow and grow over the time.
inside the gui i select automatic on/off and it writes the lines that it is uploading the workunit... after a while it stops with the networktraffic/upload the grubclient hangs there and doesnt do anything else. the .arc and the .arc.gz files are still there, same size as before and i can trigger the re-upload over and over again but nothing much happens (except for the upload traffic), but the grubclient never continues to crawl and fetch new workunits. also the only gui-buttons i can click is the "upload" button. everything else is grayed, as it thinks that is hasnt succeeded with the upload yet and wants to complete that step....
bug? is there any way to create logs? cheers.
- Hi, best way to report bugs in clients (any) is using mailing list grub-dev In case C# client - look at README.txt file, here you have mails for reports bugs. Back to report - yes, files are vanish after upload. How long you wait after upload? Sometimes server answer even after 3-4 minutes. --Thindil 16:43, 30 January 2008 (UTC)
- ok i think this is really a bug. so my datafiles in the grubng directory are for example:
01/30/2008 12:20 PM 43,820,699 username.65e519571dfa4a34d31084743fb300e7066ddde5.arc 01/30/2008 12:20 PM 22,221,625 username.65e519571dfa4a34d31084743fb300e7066ddde5.arc.gz
they are kinda big i was thinking. then i startup/restart grubng.exe. grubng.exe doesnt do anything. i have to manually activate the upload button, or set grubng to "automatic" in the "operations" menu.
then it writes out the line in the log/gui window: Sending workunit username.65e519571dfa4a34d31084743fb300e7066ddde5.arc.gz on server
i was running taskgmr.exe on windowsxp, which can measure network-interface throughput and total/sum of bytes received/sent.
grubng connects to wiki1.guest.isc.org tcport 80 (http) (checked with sysinternals process explorer) and starts sending data. but after like one to two minutes having sent only about seven (7) megabytes of data (taskmgr.exe says so) grubng.exe disconnects/finishes the http connection to wiki1.guest.isc.org and doesnt do anything. i can then click upload again at it prints the same line again, or i can toggle automatic and it resends the same data again.
and so on... it never finishes the files apparently, as it only uploads 7megabytes instead of like 22megabytes as thats the filesize of the .arc.gz file.
i am checking right now, and for the second time taskmgr.exe says that the total uploaded bytes on that network interface (the windows xp machine doesnt do anything else, testing-box) has only uploaded 15megabytes, so roughly twice the 7megabytes that its always uploading.....
bug? are my arc.gz files too big? the arc file is even bigger, with 43megs of uncompressed html-receive (?) data of the crawled webservers i guess....
so any more hints? should i delete my data files and make grubng fetch a new workunit and crawl again and try to upload the new results again?
are the datafiles too big? is the upload-size limited? has anything went wrong? cheers
so whatever i do within the gubng client, the data files always stay the same, and the grubng client doesnt do anything else. it can only wait/idle or trying to send/upload the data. it tries so, uploads about 7megabytes. finishes then. no success or any other messages are visible in the log/gui window. i can reupload/restart the upload process over and over again. nothing else ever happens. -- (2008-01-30, 01:05pm, EST)