minio-hs

A MinIO Haskell Library for Amazon S3 compatible cloud storage. https://github.com/minio/minio-hs#readme

Latest on Hackage:1.5.2

See all snapshots minio-hs appears in

Apache-2.0 licensed by MinIO Dev Team
Maintained by dev@min.io

Module documentation for 1.5.2

This version can be pinned in stack with:minio-hs-1.5.2@sha256:8c1576f0028ebeb17dcf6c2410d5ee64867c32d63074051a139d7ee5cf38a76d,9827

MinIO Client SDK for Haskell Build StatusHackageSlack

The MinIO Haskell Client SDK provides simple APIs to access MinIO and Amazon S3 compatible object storage server.

Minimum Requirements

Installation

Add to your project

Simply add minio-hs to your project’s .cabal dependencies section or if you are using hpack, to your package.yaml file as usual.

Try it out directly with ghci

From your home folder or any non-haskell project directory, just run:


stack install minio-hs

Then start an interpreter session and browse the available APIs with:


$ stack ghci
> :browse Network.Minio

Examples

The examples folder contains many examples that you can try out and use to learn and to help with developing your own projects.

Quick-Start Example - File Uploader

This example program connects to a MinIO object storage server, makes a bucket on the server and then uploads a file to the bucket.

We will use the MinIO server running at https://play.min.io in this example. Feel free to use this service for testing and development. Access credentials are present in the library and are open to the public.

FileUploader.hs

#!/usr/bin/env stack
-- stack --resolver lts-14.11 runghc --package minio-hs --package optparse-applicative --package filepath

--
-- MinIO Haskell SDK, (C) 2017-2019 MinIO, Inc.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
--     http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
--


{-# LANGUAGE OverloadedStrings   #-}
{-# LANGUAGE ScopedTypeVariables #-}
import           Network.Minio

import           Data.Monoid           ((<>))
import           Data.Text             (pack)
import           Options.Applicative
import           System.FilePath.Posix
import           UnliftIO              (throwIO, try)

import           Prelude

-- | The following example uses minio's play server at
-- https://play.min.io.  The endpoint and associated
-- credentials are provided via the libary constant,
--
-- > minioPlayCI :: ConnectInfo
--

-- optparse-applicative package based command-line parsing.
fileNameArgs :: Parser FilePath
fileNameArgs = strArgument
               (metavar "FILENAME"
                <> help "Name of file to upload to AWS S3 or a MinIO server")

cmdParser = info
            (helper <*> fileNameArgs)
            (fullDesc
             <> progDesc "FileUploader"
             <> header
             "FileUploader - a simple file-uploader program using minio-hs")

main :: IO ()
main = do
  let bucket = "my-bucket"

  -- Parse command line argument
  filepath <- execParser cmdParser
  let object = pack $ takeBaseName filepath

  res <- runMinio minioPlayCI $ do
    -- Make a bucket; catch bucket already exists exception if thrown.
    bErr <- try $ makeBucket bucket Nothing

    -- If the bucket already exists, we would get a specific
    -- `ServiceErr` exception thrown.
    case bErr of
      Left BucketAlreadyOwnedByYou -> return ()
      Left e                       -> throwIO e
      Right _                      -> return ()

    -- Upload filepath to bucket; object name is derived from filepath.
    fPutObject bucket object filepath defaultPutObjectOptions

  case res of
    Left e   -> putStrLn $ "file upload failed due to " ++ show e
    Right () -> putStrLn "file upload succeeded."

Run FileUploader

./FileUploader.hs "path/to/my/file"

Contribute

Contributors Guide

Development

To setup:

git clone https://github.com/minio/minio-hs.git

cd minio-hs/

stack install

Tests can be run with:


stack test

A section of the tests use the remote MinIO Play server at https://play.min.io by default. For library development, using this remote server maybe slow. To run the tests against a locally running MinIO live server at http://localhost:9000, just set the environment MINIO_LOCAL to any value (and unset it to switch back to Play).

To run the live server tests, set a build flag as shown below:


stack test --flag minio-hs:live-test

# OR against a local MinIO server with:

MINIO_LOCAL=1 stack test --flag minio-hs:live-test

The configured CI system always runs both test-suites for every change.

Documentation can be locally built with:


stack haddock

Changes

Changelog

Version 1.5.2

  • Fix region us-west-2 for AWS S3 (#139)
  • Build examples in CI
  • Disable live-server tests by default, but run them in CI
  • Drop support for GHC 8.2.x

Version 1.5.1

  • Add support for GHC 8.8

Version 1.5.0

  • Switch to faster map data type - all previous usage of Data.Map.Strict and Data.Set is replaced with Data.HashMap.Strict and Data.HashSet.
  • Add oiUserMetadata to parse and return user metadata stored with an object.
  • Add GetObjectResponse data type for the value returned by getObject. It now contains parsed ObjectInfo along with the conduit of object bytes.

Version 1.4.0

  • Expose runMinioRes and runMinioResWith (#129)
  • Improve Haddocks (#127)
  • Fix list objects APIs to return directory prefixes when run with recurse set to False (#126)
  • Use streaming signature for streaming payloads when on an insecure connection (#123)

Version 1.3.1

  • Add TLS helpers to check if server uses TLS, and to disable certificate validation for easier testing (#121)

Version 1.3.0

  • Retry requests that timeout using full-jitter backoff (#119)
  • Accept GetObjectOptions in statObject (#112)
  • Add encryption options to GetObjectOptions and PutObjectOptions (#111)
  • Add missing Haddock documentation (#110)
  • Add support for S3Select API (#108)
  • New travis with support for multiple GHCs (#106)
  • Fix region setting in presigned url functions (#107)

Version 1.2.0

  • Export Provider and findFirst to look for credentials (#103)

Version 1.1.0

This version brings the following changes:

  • Adds experimental Admin APIs (#88, #91, #93, #94, #95, #100)
  • Adds support for using Google Compute Storage service when S3 compatibility mode is enabled (#96, #99)

This version also brings some breaking changes (via #101):

  • Adds IsString instance to load server address, and updates initialization API to be more user friendly
  • Drops usage of data-default package and exposes explicit default values for various types used in the library.

Version 1.0.1

This version brings the following (non-breaking) changes:

  • Remove dependency on text-format (#86)
  • Remove direct dependency on exceptions (#87)
  • Adds lower-bounds on dependencies.

Version 1.0.0

This new release changes the following APIs to add new capabilities:

  • Copy Object API now supports more options for source and destination (#73)
  • get/put Object functions now support a wider set of options via a separate settings parameter (#71, #72)
  • getBucketPolicy and setBucketPolicy APIs are added (#82)
  • The library now uses UnliftIO (#83)

Version 0.3.2

This release brings the following changes:

  • Add removeIncompleteUpload API (#49)
  • Add presigned operations APIs (#56)
  • Add presigned Post Policy API (#58)
  • Skip SHA256 checksum header for secure connections (#65)
  • Remove resuming capability in PutObject (#67)
  • Add ListObjectsV1 API support (#66)
  • Add Bucket Notification APIs (#59)
  • Reverse #54 - tests fix.

Version 0.3.1

This is a bug-fix release:

  • Fix concurrency bug in limitedMapConcurrently (#53)
  • Fix tests related to listing incomplete uploads to accommodate MinIO server’s changed behaviour to not list incomplete uploads. Note that running these tests against AWS S3 are expected to fail. (#54)

Version 0.3.0

This release includes a breaking change:

Users of the library need not call runResourceT explicitly after calling runMinio. This is now done, within the runMinio call making usage a bit simpler.

Other changes:

  • Export ListUploadsResult and ListObjectsResult (#48)
    • Also take max-keys as an argument for listObjects and max-uploads for listIncompleteUploads.
  • Add bucket and object name validation (#45)
  • Add bucketExists and headBucket APIs (#42)

Version 0.2.1

  • Update dependencies, and switch to Stackage LTS 8.5

Version 0.2.0

This is an interim release which brings some new features. However, the library is not complete and APIs may change.

  • Remove listIncompleteParts API and augment listIncompleteUploads API with information about aggregate size of parts uploaded.
  • Refactors error types and simpler error throwing/handling behaviour.
  • Add removeObject API to delete objects from the service.
  • Rename Network.Minio.getService to Network.Minio.listBuckets.
  • Add docs/API.md and examples directory with comprehensive documentation and examples of high-level APIs exported by the library.
  • Rename types:
    • Rename PartInfo -> PartTuple
    • Rename ListPartInfo -> ObjectPartInfo
  • Add a bucket region cache to avoid locating a bucket’s region for every operation (mainly useful for AWS S3).
  • Add new copyObject API to perform server side object copying.
  • Rename putObjectFromSource API as putObject.
  • Separate out tests into two suites, one with a live-server and the other without any external dependencies.