amazon s3 expiration settings for buckets and folders | Report Bugs | Forum Archive

The free forums are no longer in use. It remains available as read-only archive.

Avatar
Lost password?
Advanced Search
Forum Scope


Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
The forums are currently locked and only available for read only access
sp_TopicIcon
amazon s3 expiration settings for buckets and folders
October 10, 2012
1:18 am
Avatar
Christopher
Melbourne, Australia
Member
Members
Forum Posts: 13
Member Since:
October 6, 2012
sp_UserOfflineSmall Offline

Hi Ovidiu,

Not certain this is a bug, but it certainly is for me... and I suspect, others.
If my assertion here is incorrect, please enlighten me.

Amazon s3 allows us to set expiration limits on objects within buckets e.g. backup Tar's etc.
But, at least as far as I can tell, NOT folders within buckets nor objects within folders.

When I setup XCloner with a specified "Upload Directory" in XCLoner amazon cron tab page and had a coresponding folder called "backups" in s3...
I was able to backup to that folder OK, but could not set an expiration for objects within it (nor the folder object itself).

When I left the "Upload Directory" input box empty, XCloner (I think) created a nameless folder and uploaded to that. Thus leaving me again unable to set an expiration in s3.

I would have thought XCloner should NOT create a folder if the input box is empty. If, of course, it is XCloner and not the s3 system creating such folders.

I should point out that:
1. I am ONLY using the external version - 3.1
2. I have tested manually using the "execute cron" link.
3. I have also tested with auto server side cronjob.
4. For the external setup I have successfully moved the cloner.config.php to a folder above the site root - with a couple of little hacks to admin.cloner.php and cloner.cron.php.
... But have tested without such hacks... same results.

I hope I have expressed the issue clearly.

Cheers,
Chris

October 10, 2012
8:13 am
Avatar
Ovidiu Liuta
Admin
Forum Posts: 2484
Member Since:
September 26, 2010
sp_UserOfflineSmall Offline

Hi Chris, could you post a screenshot of the folder that it's being created? As far as i know, if the s3 directory is left empty, XCloner passes that value and nothing else.

 

Ovidiu

October 10, 2012
8:15 am
Avatar
Ovidiu Liuta
Admin
Forum Posts: 2484
Member Since:
September 26, 2010
sp_UserOfflineSmall Offline

From amazon description http://aws.amazon.co.....iration/ , you can add an expiration to the folder itself, so all files inside it will be deleted when the expiration rule is meet...

October 10, 2012
8:52 am
Avatar
Christopher
Melbourne, Australia
Member
Members
Forum Posts: 13
Member Since:
October 6, 2012
sp_UserOfflineSmall Offline

Thanks for getting back Ovidiu,

Yep, I saw the article "Amazon S3 announces Object Expiration".
But I cannot make it work for me.

When I had a folder called "backups"
I tried "backups/backup_"
I tried "backups/*.tar"
I tried "backups/tar"

Suggestions for alternatives welcome 🙂

So, I rearranged everything and made buckets for backups e.g. [nhd.live.joomla.bak]
That's when the nameless folders were created by XCloner.

Also, deleting the contents of a folder is not really the solution. I need to expire, for example, backups older than 32 days... whether in a bucket or folder.

I may have a range of folders "backups", "images" etc. But I can't dynamically time-stamp folder names on the fly.

How do I post screen captures?
-- I will try this...
http://newhomedesign.com.au/cwm_pics/aws_s3_nameless_folder_example_001.jpg
http://newhomedesign.com.au/cwm_pics/aws_s3_nameless_folder_contents_002.jpg

Chris

October 10, 2012
9:15 am
Avatar
Ovidiu Liuta
Admin
Forum Posts: 2484
Member Since:
September 26, 2010
sp_UserOfflineSmall Offline

Try only backups/ within the path rule, only expired items from within that folder with get deleted!

 

Ovidiu

October 10, 2012
9:32 am
Avatar
Christopher
Melbourne, Australia
Member
Members
Forum Posts: 13
Member Since:
October 6, 2012
sp_UserOfflineSmall Offline

Thanks again Ovidiu,

I will give that a try... tomorrow.
Must go and eat and sleep or I will fall from my chair.

I will advise progress...

Chris

October 10, 2012
11:59 pm
Avatar
Christopher
Melbourne, Australia
Member
Members
Forum Posts: 13
Member Since:
October 6, 2012
sp_UserOfflineSmall Offline

Hi Ovidiu,

Yes, backups/ worked for s3 expiration.. at least items in the folder are indicating they will expire. I think it's safe to assume that will happen 🙂

I feel a bit stupid for not seeing the /... but we live and learn.

Just one more point relevant to this subject.
-- from a standalone install...
When I initiate the manual cron, a new tab opens, but it remains blank. I have noticed when initiated from Joomla or WP, there is a rolling page of actions taken... to finally announce completion. I am using FireFox 15.0.1
-- Is that a config thing that I have missed?

Thanks for your patience.

Chris

October 11, 2012
7:23 am
Avatar
Ovidiu Liuta
Admin
Forum Posts: 2484
Member Since:
September 26, 2010
sp_UserOfflineSmall Offline

You probably have some kind of error, the cron script should display at least some text. Try running it from the server console if you have access and see how it goes, also enable php errors display to get any error messages that might come out of it...

 

Ovidiu

Forum Timezone: America/Chicago
Most Users Ever Online: 867
Currently Online:
Guest(s) 1
Currently Browsing this Page:
1 Guest(s)
Top Posters:
mlguru: 30
Django29: 29
Andy: 21
D: 21
Marcus: 20
Jamie F: 19
Member Stats:
Guest Posters: 738
Members: 10030
Moderators: 2
Admins: 3
Forum Stats:
Groups: 3
Forums: 7
Topics: 2397
Posts: 8236
Newest Members:
Omar Vera
Moderators: TriP: 0, Steve Burge: 0
Administrators: Ovidiu Liuta: 2484, Victor Drover: 1, Valentin Barbu: 0