Adding too many functions to object storage makes the result look much more like POSIX. In either case, the interesting part is trying to find the balance between object-oriented storage and POSIX. An obvious starting point is to either add the object storage functions (GET/PUT/DELETE) to POSIX, or to add some POSIX functions to object storage. What really is needed is a hybrid file system that contains some of the aspects of object storage and some of the aspects of a POSIX file system. However, I think it might be possible to combine object-storage and POSIX to create something that comes close to the ideal. People using POSIX storage would like things to be simpler, approaching that of object storage, and they would like the cost to come down.Īt this point, I would normally insert a picture of the rainbow unikitty with butterfly wings and say good luck. Typically POSIX storage is faster than object storage but also a bit more expensive.Īs a result, people using object storage want faster performance and byte-level access while retaining the simplicity, ease of use, and the cost. However, with the large number of IO functions comes complexity, both for the application and the file system. The POSIX compliance provides a wide range of IO functions for applications to use, including byte-level access. POSIX file systems are the most common storage system in use today. On the other hand, people complain that applications have to be rewritten to use the storage and that they are slower than other forms of storage. Object storage systems are popular for many reasons, most notably simplicity, ease of use, and cost (at least compared to NAS systems). It also has some additional features unique to s3ql, including the following: It handles hardlinks, symlinks, standard permissions, extended attributes and file sizes up to 2TB. It is POSIX-compliant so applications can run using the storage without worry. It acts as an intermediate file system between S3 and local applications. One of the more fully featured tools is s3ql. There are many, many more questions about these intermediaries, but in many cases they are the best solution available to handle data movement to/from the object storage for you. How often do you synchronize data from the local file system to the object storage? What if the POSIX application is doing lots of small byte IO with the data files on the local storage? How and when does the intermediary move the data to/from the object storage? What happens if the local storage crashes? How is data consistency insured? The concept of the intermediaries is good, but the devil is always in the details. The intermediaries then copy the data to/from the object storage systems. The POSIX applications interact with the data on the local file system (which is POSIX-compliant). These solutions GET/PUT complete files from the object storage to a local file system. There are some object-storage intermediaries that provide a gateway to/from the object-storage system and POSIX-compliant applications. This means you have to port or rewrite your application to adapt to the storage paradigm. Your application has to handle the data in the file. When you read a file, you get the entire file via the GET function. You can’t do any byte-level file access like you can do with POSIX. Object-oriented storage systems are really an all-or-nothing situation. This is only a partial list of POSIX IO functions including some of the extended attributes. Below is a partial list of IO functions for Linux: On the other hand, tried-and-true POSIX file systems that have a massive range of data access functions. The POST and HEAD commands are not universal, but you do see them in object storage systems that are compatible with AWS S3.
0 Comments
Leave a Reply. |