Merge branch 'feature/python3' into develop

CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## vX.X.X
+
+### Breaking changes
+Now work with python 3! Support of python 2 is no longer available.  
+You should now use python 3 in order to use prismedia
+
+### Features
+ - Add a requirements.txt file to make installing requirement easier.  
+ - Add a debug option to show some infos before uploading (thanks to @zykino)  
+ - Now uploading to Peertube before Youtube (thanks to @zykino)
+
 ## v0.7.1
 
 ### Fixes

README.md

@@ -1,20 +1,28 @@
 # Prismedia
 
-A scripting way to upload videos to peertube and youtube written in python2
+Scripting your way to upload videos to peertube and youtube. Works with Python 3.5+.
 
 ## Dependencies
-Search in your package manager, otherwise use ``pip install --upgrade``
+Search in your package manager, or with `pip` use ``pip install -r requirements.txt``
+ - configparser
+ - docopt
+ - future
+ - google-api-python-client
  - google-auth
- - google-auth-oauthlib
  - google-auth-httplib2
- - google-api-python-client
- - docopt
- - schema
+ - google-auth-oauthlib
+ - httplib2
+ - oauthlib
  - python-magic
- - python-magic-bin
+ - python-magic-bin (Windows only)
+ - requests
+ - requests-oauthlib
  - requests-toolbelt
+ - schema
  - tzlocal
- - unidecode
+ - Unidecode
+ - uritemplate
+ - urllib3
 
 ## Configuration
 
@@ -39,18 +47,16 @@ Prismedia will try to use this file at each launch, and re-ask for authenticatio
 The default youtube_secret.json should allow you to upload some videos.  
 If you plan an larger usage, please consider creating your own youtube_secret file:
 
-- Go to the [Google console](https://console.developers.google.com/).
-- Create project.
-- Side menu: APIs & auth -> APIs
-- Top menu: Enabled API(s): Enable all Youtube APIs.
-- Side menu: APIs & auth -> Credentials.
-- Create a Client ID: Add credentials -> OAuth 2.0 Client ID -> Other -> Name: prismedia1 -> Create -> OK
-- Download JSON: Under the section "OAuth 2.0 client IDs". Save the file to your local system.
-- Save this JSON as your youtube_secret.json file.
+ - Go to the [Google console](https://console.developers.google.com/).
+ - Create project.
+ - Side menu: APIs & auth -> APIs
+ - Top menu: Enabled API(s): Enable all Youtube APIs.
+ - Side menu: APIs & auth -> Credentials.
+ - Create a Client ID: Add credentials -> OAuth 2.0 Client ID -> Other -> Name: prismedia1 -> Create -> OK
+ - Download JSON: Under the section "OAuth 2.0 client IDs". Save the file to your local system.
+ - Save this JSON as your youtube_secret.json file.
 
 ## How To
-Currently in heavy development
-
 Support only mp4 for cross compatibility between Youtube and Peertube
 
 Simply upload a video:
@@ -86,11 +92,11 @@ Use --help to get all available options:
 Options:
   -f, --file=STRING Path to the video file to upload in mp4
   --name=NAME  Name of the video to upload. (default to video filename)
+  --debug  Trigger some debug information like options used (default: no)
   -d, --description=STRING  Description of the video. (default: default description)
   -t, --tags=STRING  Tags for the video. comma separated.
-                     WARN: tags with space and special characters (!, ', ", ?, ...)
+                     WARN: tags with punctuation (!, ', ", ?, ...)
                            are not supported by Mastodon to be published from Peertube
-                           use mastodon compatibility below
   -c, --category=STRING  Category for the videos, see below. (default: Films)
   --cca  License should be CreativeCommon Attribution (affects Youtube upload only)
   -p, --privacy=STRING  Choose between public, unlisted or private. (default: private)
@@ -106,7 +112,6 @@ Options:
   --publishAt=DATE  Publish the video at the given DATE using local server timezone.
                     DATE should be on the form YYYY-MM-DDThh:mm:ss eg: 2018-03-12T19:00:00
                     DATE should be in the future
-                    For Peertube, requires the "atd" and "curl utilities installed on the system
   --thumbnail=STRING    Path to a file to use as a thumbnail for the video.
                         Supported types are jpg and jpeg.
                         By default, prismedia search for an image based on video name followed by .jpg or .jpeg
@@ -114,7 +119,7 @@ Options:
                     If the channel is not found, spawn an error except if --channelCreate is set.
   --channelCreate  Create the channel if not exists. (Peertube only, default do not create)
                    Only relevant if --channel is set.
-  --playlist=STRING Set the playlist to use for the video. Also known as Channel for Peertube.
+  --playlist=STRING Set the playlist to use for the video.
                     If the playlist is not found, spawn an error except if --playlistCreate is set.
   --playlistCreate  Create the playlist if not exists. (default do not create)
                     Only relevant if --playlist is set.
@@ -155,16 +160,18 @@ Languages:
   - [x] add videos to playlist
   - [x] create playlist
   - [x] schedule your video with publishAt
-  - [x] combine channel and playlist (Peertube only as channel is Peertube feature). See [issue 40](https://git.lecygnenoir.info/LecygneNoir/prismedia/issues/40 for detailed usage.
+  - [x] combine channel and playlist (Peertube only as channel is Peertube feature). See [issue 40](https://git.lecygnenoir.info/LecygneNoir/prismedia/issues/40) for detailed usage.
 - [x] Use a config file (NFO) file to retrieve videos arguments
 - [x] Allow to choose peertube or youtube upload (to resume failed upload for example)
-- [ ] Record and forget: put the video in a directory, and the script uploads it for you
-- [ ] Usable on Desktop (Linux and/or Windows and/or MacOS)
-- [ ] Graphical User Interface
+- [x] Usable on Desktop (Linux and/or Windows and/or MacOS)
 
 ## Compatibility
 
-If your server uses peertube before 1.0.0-beta4, use the version inside tag 1.0.0-beta3!
+ - If you still use python2, use the version 0.7.1 (no more updated)
+ - peertube before 1.0.0-beta4, use the version inside tag 1.0.0-beta3
 
 ## Sources
 inspired by [peeror](https://git.rigelk.eu/rigelk/peeror) and [youtube-upload](https://github.com/tokland/youtube-upload)
+
+## Contributors
+Thanks to: @Zykino, @meewan, @rigelk 😘

lib/pt_upload.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python
 # coding: utf-8
 
 import os
@@ -10,7 +10,7 @@ import pytz
 from os.path import splitext, basename, abspath
 from tzlocal import get_localzone
 
-from ConfigParser import RawConfigParser
+from configparser import RawConfigParser
 from requests_oauthlib import OAuth2Session
 from oauthlib.oauth2 import LegacyApplicationClient
 from requests_toolbelt.multipart.encoder import MultipartEncoder
@@ -57,7 +57,7 @@ def get_default_channel(user_info):
 
 def get_channel_by_name(user_info, options):
     for channel in user_info["videoChannels"]:
-        if channel['displayName'].encode('utf8') == str(options.get('--channel')):
+        if channel['displayName'] == options.get('--channel'):
             return channel['id']
 
 
@@ -67,16 +67,17 @@ def create_channel(oauth, url, options):
     channel_name = utils.cleanString(str(options.get('--channel')))
     # Peertube allows 20 chars max for channel name
     channel_name = channel_name[:19]
-    data = '{"name":"' + channel_name +'", \
-            "displayName":"' + str(options.get('--channel')) +'", \
-            "description":null}'
+    data = '{"name":"' + channel_name + '", \
+            "displayName":"' + options.get('--channel') + '", \
+            "description":null, \
+            "support":null}'
 
     headers = {
-        'Content-Type': "application/json"
+        'Content-Type': "application/json; charset=UTF-8"
     }
     try:
         response = oauth.post(url + "/api/v1/video-channels/",
-                              data=data,
+                              data=data.encode('utf-8'),
                               headers=headers)
     except Exception as e:
         if hasattr(e, 'message'):
@@ -89,8 +90,10 @@ def create_channel(oauth, url, options):
             jresponse = jresponse['videoChannel']
             return jresponse['id']
         if response.status_code == 409:
-            logging.error('Peertube: Error: It seems there is a conflict with an existing channel, please beware '
-                          'Peertube internal name is compiled from 20 firsts characters of channel name.'
+            logging.error('Peertube: Error: It seems there is a conflict with an existing channel named '
+                          + channel_name + '.'
+                          ' Please beware Peertube internal name is compiled from 20 firsts characters of channel name.'
+                          ' Also note that channel name are not case sensitive (no uppercase nor accent)'
                           ' Please check your channel name and retry.')
             exit(1)
         else:
@@ -105,7 +108,7 @@ def get_default_playlist(user_info):
 
 def get_playlist_by_name(user_playlists, options):
     for playlist in user_playlists["data"]:
-        if playlist['displayName'].encode('utf8') == str(options.get('--playlist')):
+        if playlist['displayName'] == options.get('--playlist'):
             return playlist['id']
 
 

lib/utils.py

@@ -1,7 +1,7 @@
 #!/usr/bin/python
 # coding: utf-8
 
-from ConfigParser import RawConfigParser, NoOptionError, NoSectionError
+from configparser import RawConfigParser, NoOptionError, NoSectionError
 from os.path import dirname, splitext, basename, isfile
 import re
 from os import devnull
@@ -102,7 +102,7 @@ def getLanguage(language, platform):
 def remove_empty_kwargs(**kwargs):
     good_kwargs = {}
     if kwargs is not None:
-        for key, value in kwargs.iteritems():
+        for key, value in kwargs.items():
             if value:
                 good_kwargs[key] = value
     return good_kwargs
@@ -132,7 +132,7 @@ def loadNFO(options):
             logging.info("Using " + options.get('--nfo') + " as NFO, loading...")
             if isfile(options.get('--nfo')):
                 nfo = RawConfigParser()
-                nfo.read(options.get('--nfo'))
+                nfo.read(options.get('--nfo'), encoding='utf-8')
                 return nfo
             else:
                 logging.error("Given NFO file does not exist, please check your path.")
@@ -147,7 +147,7 @@ def loadNFO(options):
                 try:
                     logging.info("Using " + nfo_file + " as NFO, loading...")
                     nfo = RawConfigParser()
-                    nfo.read(nfo_file)
+                    nfo.read(nfo_file, encoding='utf-8')
                     return nfo
                 except Exception as e:
                     logging.error("Problem with NFO file: " + str(e))
@@ -160,7 +160,7 @@ def loadNFO(options):
         try:
             logging.info("Using " + nfo_file + " as NFO, loading...")
             nfo = RawConfigParser()
-            nfo.read(nfo_file)
+            nfo.read(nfo_file, encoding='utf-8')
             return nfo
         except Exception as e:
             logging.error("Problem with nfo file: " + str(e))
@@ -172,7 +172,7 @@ def parseNFO(options):
     nfo = loadNFO(options)
     if nfo:
         # We need to check all options and replace it with the nfo value if not defined (None or False)
-        for key, value in options.iteritems():
+        for key, value in options.items():
             key = key.replace("-", "")
             try:
                 # get string options
@@ -192,22 +192,7 @@ def upcaseFirstLetter(s):
     return s[0].upper() + s[1:]
 
 def cleanString(toclean):
-    toclean = toclean.decode('utf-8')
     toclean = unidecode.unidecode(toclean)
     cleaned = re.sub('[^A-Za-z0-9]+', '', toclean)
 
     return cleaned
-
-def decodeArgumentStrings(options, encoding):
-    # Python crash when decoding from UTF-8 to UTF-8, so we prevent this
-    if "utf-8" == encoding.lower():
-        return;
-
-    if options["--name"] is not None:
-        options["--name"] = options["--name"].decode(encoding)
-
-    if options["--description"] is not None:
-        options["--description"] = options["--description"].decode(encoding)
-
-    if options["--tags"] is not None:
-        options["--tags"] = options["--tags"].decode(encoding)

lib/yt_upload.py

@@ -1,8 +1,8 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python
 # coding: utf-8
 # From Youtube samples : https://raw.githubusercontent.com/youtube/api-samples/master/python/upload_video.py  # noqa
 
-import httplib
+import http.client
 import httplib2
 import random
 import time
@@ -38,13 +38,13 @@ MAX_RETRIES = 10
 RETRIABLE_EXCEPTIONS = (
     IOError,
     httplib2.HttpLib2Error,
-    httplib.NotConnected,
-    httplib.IncompleteRead,
-    httplib.ImproperConnectionState,
-    httplib.CannotSendRequest,
-    httplib.CannotSendHeader,
-    httplib.ResponseNotReady,
-    httplib.BadStatusLine,
+    http.client.NotConnected,
+    http.client.IncompleteRead,
+    http.client.ImproperConnectionState,
+    http.client.CannotSendRequest,
+    http.client.CannotSendHeader,
+    http.client.ResponseNotReady,
+    http.client.BadStatusLine,
 )
 
 RETRIABLE_STATUS_CODES = [500, 502, 503, 504]
@@ -146,7 +146,7 @@ def initialize_upload(youtube, options):
 
     # Call the API's videos.insert method to create and upload the video.
     insert_request = youtube.videos().insert(
-        part=','.join(body.keys()),
+        part=','.join(list(body.keys())),
         body=body,
         media_body=MediaFileUpload(path, chunksize=-1, resumable=True)
     )
@@ -168,7 +168,7 @@ def get_playlist_by_name(youtube, playlist_name):
         maxResults=50
     ).execute()
     for playlist in response["items"]:
-        if playlist["snippet"]['title'].encode('utf8') == str(playlist_name):
+        if playlist["snippet"]['title'] == playlist_name:
             return playlist['id']
 
 

prismedia_upload.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python2
+#!/usr/bin/env python3
 # coding: utf-8
 
 """
@@ -13,11 +13,11 @@ Usage:
 Options:
   -f, --file=STRING Path to the video file to upload in mp4
   --name=NAME  Name of the video to upload. (default to video filename)
+  --debug  Trigger some debug information like options used (default: no)
   -d, --description=STRING  Description of the video. (default: default description)
   -t, --tags=STRING  Tags for the video. comma separated.
-                     WARN: tags with space and special characters (!, ', ", ?, ...)
+                     WARN: tags with punctuation (!, ', ", ?, ...)
                            are not supported by Mastodon to be published from Peertube
-                           use mastodon compatibility below
   -c, --category=STRING  Category for the videos, see below. (default: Films)
   --cca  License should be CreativeCommon Attribution (affects Youtube upload only)
   -p, --privacy=STRING  Choose between public, unlisted or private. (default: private)
@@ -33,7 +33,6 @@ Options:
   --publishAt=DATE  Publish the video at the given DATE using local server timezone.
                     DATE should be on the form YYYY-MM-DDThh:mm:ss eg: 2018-03-12T19:00:00
                     DATE should be in the future
-                    For Peertube, requires the "atd" and "curl utilities installed on the system
   --thumbnail=STRING    Path to a file to use as a thumbnail for the video.
                         Supported types are jpg and jpeg.
                         By default, prismedia search for an image based on video name followed by .jpg or .jpeg
@@ -41,7 +40,7 @@ Options:
                     If the channel is not found, spawn an error except if --channelCreate is set.
   --channelCreate  Create the channel if not exists. (Peertube only, default do not create)
                    Only relevant if --channel is set.
-  --playlist=STRING Set the playlist to use for the video. Also known as Channel for Peertube.
+  --playlist=STRING Set the playlist to use for the video.
                     If the playlist is not found, spawn an error except if --playlistCreate is set.
   --playlistCreate  Create the playlist if not exists. (default do not create)
                     Only relevant if --playlist is set.
@@ -97,6 +96,9 @@ except ImportError:
                   'see https://github.com/ahupp/python-magic\n')
     exit(1)
 
+if sys.version_info[0] < 3:
+    raise Exception("Python 3 or a more recent version is required.")
+
 VERSION = "prismedia v0.7.1"
 
 VALID_PRIVACY_STATUSES = ('public', 'private', 'unlisted')
@@ -107,7 +109,7 @@ VALID_CATEGORIES = (
     "how to", "education", "activism", "science & technology",
     "science", "technology", "animals"
 )
-VALID_PLATFORM = ('youtube', 'peertube')
+VALID_PLATFORM = ('youtube', 'peertube', 'none')
 VALID_LANGUAGES = ('arabic', 'english', 'french',
                    'german', 'hindi', 'italian',
                    'japanese', 'korean', 'mandarin',
@@ -170,17 +172,17 @@ if __name__ == '__main__':
     schema = Schema({
         '--file': And(str, validateVideo, error='file is not supported, please use mp4'),
         Optional('--name'): Or(None, And(
-                                basestring,
+                                str,
                                 lambda x: not x.isdigit(),
                                 error="The video name should be a string")
                                ),
         Optional('--description'): Or(None, And(
-                                        basestring,
+                                        str,
                                         lambda x: not x.isdigit(),
                                         error="The video description should be a string")
                                       ),
         Optional('--tags'): Or(None, And(
-                                    basestring,
+                                    str,
                                     lambda x: not x.isdigit(),
                                     error="Tags should be a string")
                                ),
@@ -206,6 +208,7 @@ if __name__ == '__main__':
                                     validatePublish,
                                     error="DATE should be the form YYYY-MM-DDThh:mm:ss and has to be in the future")
                                     ),
+        Optional('--debug'): bool,
         Optional('--cca'): bool,
         Optional('--disable-comments'): bool,
         Optional('--nsfw'): bool,
@@ -220,7 +223,6 @@ if __name__ == '__main__':
         '--version': bool
     })
 
-    utils.decodeArgumentStrings(options, locale.getpreferredencoding())
     options = utils.parseNFO(options)
 
     if not options.get('--thumbnail'):
@@ -231,7 +233,11 @@ if __name__ == '__main__':
     except SchemaError as e:
         exit(e)
 
-    if options.get('--platform') is None or "youtube" in options.get('--platform'):
-        yt_upload.run(options)
+    if options.get('--debug'):
+        print(sys.version)
+        print(options)
+
     if options.get('--platform') is None or "peertube" in options.get('--platform'):
         pt_upload.run(options)
+    if options.get('--platform') is None or "youtube" in options.get('--platform'):
+        yt_upload.run(options)

requirements.txt

@@ -0,0 +1,19 @@
+configparser
+docopt
+future
+google-api-python-client
+google-auth
+google-auth-httplib2
+google-auth-oauthlib
+httplib2
+oauthlib
+python-magic
+python-magic-bin; platform_system == "Windows"
+requests
+requests-oauthlib
+requests-toolbelt
+schema
+tzlocal
+Unidecode
+uritemplate
+urllib3