Computing the BigQuery cost of a single Airflow DAG run
TL;DR To estimate the number of bytes processed by a single run of a given DAG on a given day, execute the following: ```sh $ pip3 install https://github.com/dtws/[email protected] $ airf
OpenVPN Summary + Amazon Linux2
# Overview# --OpenVPN encapsulates L2 and L3 protocols such as IP / TCP with SSL / TLS and securely connects between the OpenVPN server and the OpenVPN client. --OpenVPN uses PKI. --Files requir
Put the second axis in 2dhistgram of matplotlib
## Example: Add a second axis to the y-axis  A, B, and C are added to the
Machine learning engineer lawyer explains AI and rights story
Hello everyone. This is the article on the 23rd day of "[ABEJA Advent Calendar 2020](https://qiita.com/advent-calendar/2020/abeja)". # Introduction While belonging to a law firm as a lawyer, I su
I made CORS custom middleware with Django
This article is the 24th day article of [Iwate University Advent Calendar 2020](https://qiita.com/advent-calendar/2020/iwate-u). # Introduction There was a time when I created an API using Django f
[2021 version] Python installation Windows 10 (64bit) edition
# Introduction Since there are many opportunities to install Python, I made it a manual. This article is for Windows 10 (64bit). # Python3.9 installation Go to the official Python download page (h
Build a Docker image containing the private repository Python library on GitHub Actions
# background purpose I've summarized how to bring an internal library from another private repository when building a Python image on Github Actions. I already had an article by my colleague @ ely
[Python, Excel] Eliminate cell merging
# Lamentation ** Why is there something called "cell merging" in this world ... ** Because it looks good? Can you put together the same numbers? Is it because it shifts when printing? ** Please ma
Transcendental simple and clear! !! Difference between single quotes and double quotes in Python
Conclusion: no difference I was worried because it will be used properly in the sample code etc., but it seems that there is no particular difference related to the system. Since I am using a JI
LowMemoryKiller ~ How Android Activity is destroyed ~
This article is the 23rd day article of [LIFULL Part 2 Advent Calendar 2020](https://qiita.com/advent-calendar/2020/lifull2). : christmas_tree: This time, I'll take up the low-layer story of Androi
To those who do not have her on Christmas Eve ◯ Voice keyboard
Hello, everyone at the limit development mackerel of dangerous people or public relations and blog FascodeNetwork is to put a thing spider you're doing, such as such as the fat active. Please note t
Linux read-ahead caching read_ahead_kb
This article is too interesting! > # Lookahead If the required data is read synchronously every time a user requests it, it takes a long time to wait for disk I/O. Therefore, the cache is enriched
Try hitting the YouTube API in Python
> This article is the 25th day of [** Fukushima National College of Technology Advent Calendar 2020 **](https://qiita.com/advent-calendar/2020/nitfc). This article is about API. Please do not abuse
Regularly upload files to Google Drive using the Google Drive API in Python
# background I want a script for a job that regularly uploads files to google drive with python. If you search, there are many articles that use PyDrive, but PyDrive is not well maintained, and it s
About Opencv ①
Notes about Opencv3 series Basically, it is a thing while checking the official document. #### environment windows10 home 64bit Python 3.7.9 opencv 3.4.2.17 Jupyter Notebook used (Anaconda not us
Addition of fixed processing when starting python interpreter
# environment - windows 10 pro - python 3.9.0 # Purpose Because it was troublesome to write an import statement every time I started it with the python interpreter I investigated the setting method
Random number generation according to multivariate normal distribution using Cholesky decomposition of covariance matrix
<h2> Table of contents </h2> <ol> <li><a href="#adv">Advent Calendar 2020/12/10</a></li> <li> <a href="#mnbase"> Multivariate normal distribution </a> </li> <li> <a href="#binorm"> Rand
A story about a 503 error on Heroku open
This is the first post. Even if it is too long, I will get tired, so I write it in rough sentences. Please note that it may be difficult to read. This article is based on the article <a href= "https
Try writing JSON format data to object storage Cloudian/S3
## Introduction You can use the AWS SDK for Python (boto3) to access Cloudian and S3 with almost no changes to your program. I hope it will be a reference for those who want to use object storage in
How to install Go on Ubuntu
# Check the latest version [https://golang.org/dl/](https://golang.org/dl/) # Installation ``` cd ~ wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz sudo tar -C /usr/local -xzf go1.15.6.linu
go: embed Detailed explanation --Usage-
This article is for the 23rd day of [Go Advent Calendar 2020](https://qiita.com/advent-calendar/2020/go) and the 24th day of [CyberAgent Developers Advent Calendar 2020](https://adventar.org/calendar
Bookkeeping Learned with Python-The Flow of Bookkeeping-
## Introduction The purpose of this article is to code and understand the flow of bookkeeping in Python (mainly Pandas). Since the purpose is to understand the general flow, some items that appear
Convert pandas dataframe elements to regular string type
``` res = df.loc[ <Index name>, <Column name> ] ``` You can do it. Note that it cannot be done with `df.loc [<index name>, [<column name>]]`.
Try to extract specific data from JSON format data in object storage Cloudian/S3
## Introduction You can use the AWS SDK for Python (boto3) to access Cloudian and S3 with almost no changes to your program. I hope it will be a reference for those who want to use object storage in
[Logistic regression] Implement holdout verification with stats models
# Introduction In python, scikit-learn and statsmodels are mainly used as libraries that can use logistic regression models. While statsmodels has advantages that scikit-learn does not have, such as
Create "operation log" CSV formatting tool in 5 days with Python Pandas PyInstaller
## A record of making one tool in 5 days without punch lines or mountains I want to do something new. What you can use at work anyway → I wonder if I should shape the OPLOG. Oplog is our product t
[Metashape] Obtained 3D coordinates of specific points of individual images-Implemented from Python API with reference to Agisoft forum-
# What is Metashape? Since I am doing image processing for drones in my master's research, is it a 3D construction software called Metashape (formerly Photoscan)? I am using. Originally, image proc
Python Math Series ② Matrix Multiplication
## Background I'm reading [Deep Learning Book](https://www.amazon.co.jp/%E6%B7%B1%E5%B1%A4%E5%AD%A6%E7%BF%92-Ian-Goodfellow/dp/4048930621), but I get sleepy when I just read the characters, so I'll t
Algorithm learned with Python 10th: Binary search
# #Algorithm learned in Python <Binary search> ## Introduction Implement the basic algorithm in Python to deepen your understanding of the algorithm. Binary search is dealt with as the 10th bullet.
[Python] Let's throw school tasks into Python [Efficiency]
# Introduction I'm currently in my first year of high school In my spare time due to the corona sickness, I studied Python thoroughly and I started studying Java and Elixir all night, so my school