6 replies to this topic

[chaosowns]

Hello,

I have been asked to try to program a ftp file stream in python. Here's the complete story:

There's a FTP server with a comma seperated file (csv) on it. A device writes log information in to the csv file whenever something log-worthy occures. This csv file is around 22 MB and keeps on growing. I want to read the latest csv entry that's gotten in to the file. So my question is:

Is it possible to connect to the FTP server, open up a stream to the csv file and retrieve the latest data? I have to able to get a constant stream and always get the latest csv entry.

If this is possible wont there be a conflict when the device is writing to the .csv file and i'm reading from it?

If this isn't possible in python, do you have any idea if it is in another programming language?

Hope i explained everything well enough.

Thanks in advance.

[Lord Cat]

Wait are you saying you want to Read a file thats not yours?
Do you have any reason for looking at the file?

Or do you own the computer that contains the file

[chaosowns]

Wait are you saying you want to Read a file thats not yours?
Do you have any reason for looking at the file?

Or do you own the computer that contains the file

Thx for your reply.

It's an FTP server where the file is stored with authorisation (of course), i got the username and pass to make a ftp connection.

Let me explain again:
Say you have a device somewhere in a network that logs every handling it does in a file. That file is downloadable by using FTP. I have to check if something odd happens to the device by checking the log file. If i would download the entire log file everytime to check if something has happened my process will be slow and inefficient because the log file is big. So if possible i would want to open a stream to the file and get the latest log information. But i dont know if it's possible in python...i hope so.

[Lord Cat]

Oh...Well would it be possible for the file you need to see have set dates on it everytime it logs something you like a date stamp then you can just recall the date and it will only give you that part...If its something like that then you should be able to do something like with python but as the thing says i'm a newbie and have only been learning for the past few months

[chaosowns]

Oh...Well would it be possible for the file you need to see have set dates on it everytime it logs something you like a date stamp then you can just recall the date and it will only give you that part...If its something like that then you should be able to do something like with python but as the thing says i'm a newbie and have only been learning for the past few months

The csv file has dates in it, every csv entry starts with a date and then some log info. You have any idea how i will get only a specific part out of the file in python?

[TkTech]

The FTP protocol does not support resuming a file transfer from a set position. It supports RESTART, but it'll only work within the same session as the initial download. There may be some extension I don't know about.

[tbohon]

I handle ftp work for my employer as a major part of my job and I can tell you that what you want to do isn't defined in the ftp protocol and therefore can't be done. There are two ways to do what you want that I can see.

Option 1 is to pull the entire file down and use the 'tail' utility (converted from Unix to Windows) to pull the last line of the file. Depending on how often you want to do this and how fast your connection is this is probably going to be very, very painful.

Option 2 would be to install the 'tail' utility on the server and set it up to pull the last line as a scheduled job (every day, every hour, every 15 minutes, etc.) and send the output to a temporary file. Your ftp job could then connect a minute or two later, pull that temporary file and delete it after you have a copy.

I'd vote for Option 2 as the 'tail' command will operate in seconds and the ftp connection/pull/delete operation will likely be less than a minute.

If you have any other questions feel free to ask - as I said I've been doing ftp work x10+ years and in fact was tasked with writing an ftp scripting language and ftp engine which is in use company-wide.