Stock Prices Prediction Using Machine Learning and Deep Learning (2024)

Explore
- Discover
  BlogsUnpacking the latest trends in AI - A knowledge capsuleLeadership PodcastsKnow the perspective of top leaders
  Expert SessionsGo deep with industry leaders in live, interactive sessionsComprehensive GuidesMaster complex topics with comprehensive, step-by-step resources
- Learn
  Free CoursesKickstart your AI journey with our comprehensive courses
  Learning PathsExpert-curated paths for every goalGen AI Pinnacle ProgramLearn cutting-edge skills for your AI career
- Engage
  Thriving CommunityShare knowledge, spark ideas, and build connectionsExciting EventsDive into the hottest trends and network with industry leaders
  Challenging HackathonsTest your skills, unleash your creativity, and win big DataHack Summit 2024India's Most Futuristic GenAI Conference
- Contribute
  Become a WordsmithShare your voice, and win big in blogathonsBecome a MentorCraft careers by sharing your knowledge
  Become a SpeakerInspire minds, share your expertiseBecome an InstructorShape next-gen innovators through our programs
- Corporate
  Our OfferingsBuild a data-powered and data-driven workforceTrainingsBridge your team's data skills with targeted training
  Analytics maturityUnleash the power of analytics for smarter outcomesData CultureBreak down barriers and democratize data access and usage

- Logout

Aishwarya Singh02 May, 2023 • 12 min read

17 minutes

⭐⭐⭐⭐⭐

Rating: 5 out of 5.

Introduction

Predicting how the stock market will perform is one of the most difficult things to do. There are so many factors involved in the prediction – physical factors vs. psychological, rational and irrational behavior, etc. All these aspects combine to make share prices volatile and very difficult to predict with a high degree of accuracy.

Can we use machine learning as a game-changer in this domain? Using features like the latest announcements about an organization, their quarterly revenue results, etc., machine learning techniques have the potential to unearth patterns and insights we didn’t see before, and these can be used to make unerringly accurate predictions.

In this article, we will work with historical data about the stock prices of a publicly listed company. We will implement a mix of machine learning algorithms to predict the future stock price of this company, starting with simple algorithms like averaging and linear regression, and then move on to advanced techniques like Auto ARIMA and LSTM. The above-stated machine learning algorithms can be easily learned from this ML Course online.

The core idea behind this article is to showcase how these algorithms are implemented. I will briefly describe the technique and provide relevant links to brush up on the concepts as and when necessary. In case you’re a newcomer to the world of time series,I suggest going through the following articles first:

A comprehensive beginner’s guide to create a Time Series Forecast
A Complete Tutorial on Time Series Modeling
Free Course: Time Series Forecasting using Python

Are you a beginner looking for a place to start your data science journey? Presenting a comprehensive course, full of knowledge and data science learning, curated just for you! This course covers everything from basics of Machine Learning to Advanced concepts of ML, Deep Learning and Time series.

Introduction
Understanding the Problem Statement
Moving Average
Linear Regression
k-Nearest Neighbours
Auto ARIMA
Prophet
Long Short Term Memory (LSTM)
Conclusion
Frequently Asked Questions

Understanding the Problem Statement

We’ll dive into the implementation part of this article soon, but first it’s important to establish what we’re aiming to solve. Broadly, stock market analysis is divided into two parts – Fundamental Analysis and Technical Analysis.

Fundamental Analysis involves analyzing the company’s future profitability on the basis of its current business environment and financial performance.
Technical Analysis, on the other hand, includes reading the charts and using statistical figures to identify the trends in the stock market.

As you might have guessed, our focus will be on the technical analysis part. We’ll be using a dataset fromQuandl(you can find historical data for various stocks here) and for this particular project, I have used the data for ‘Tata Global Beverages’. Time to dive in!

Note: Here is the dataset I used for the code: Download

We will first load the dataset and define the target variable for the problem:

Python Code:

There are multiple variables in the dataset – date, open, high, low, last, close, total_trade_quantity, and turnover.

The columns Open and Close represent the starting and final price at which the stock is traded on a particular day.
High, Low and Last represent the maximum, minimum, and last price of the share for the day.
Total Trade Quantity is the number of shares bought or sold in the day and Turnover (Lacs) is the turnover of the particular company on a given date.

Another important thing to note is that the market is closed on weekends and public holidays. Notice the above table again, some date values are missing – 2/10/2018, 6/10/2018, 7/10/2018. Of these dates, 2nd is a national holiday while 6th and 7th fall on a weekend.

The profit or loss calculation is usually determined by the closing price of a stock for the day, hence we will consider the closing price as the target variable. Let’s plot the target variable to understand how it’s shaping up in our data:

#setting index as datedf['Date'] = pd.to_datetime(df.Date,format='%Y-%m-%d')df.index = df['Date']#plotplt.figure(figsize=(16,8))plt.plot(df['Close'], label='Close Price history')

Stock Prices Prediction Using Machine Learning and Deep Learning (10)

In the upcoming sections, we will explore these variables and use different techniques to predict the daily closing price of the stock.

Moving Average

Introduction

‘Average’ is easily one of the most common things we use in our day-to-day lives. For instance, calculating the average marks to determine overall performance, or finding the average temperature of the past few days to get an idea about today’s temperature – these all are routine tasks we do on a regular basis. So this is a good starting point to use on our dataset for making predictions.

The predicted closing price for each day will be the average of a set of previously observed values. Instead of using the simple average, we will be using the moving average technique which uses the latest set of values for each prediction. In other words, for each subsequent step, the predicted values are taken into consideration while removing the oldest observed value from the set. Here is a simple figure that will help you understand this with more clarity.

Stock Prices Prediction Using Machine Learning and Deep Learning (11)

We will implement this technique on our dataset. The first step is to create a dataframe that contains only the Date and Close price columns, then split it into train and validation sets to verify our predictions.

Implementation

Just checking the RMSE does not help us in understanding how the model performed. Let’s visualize this to get a more intuitive understanding. So here is a plot of the predicted values along with the actual values.

#plotvalid['Predictions'] = 0valid['Predictions'] = predsplt.plot(train['Close'])plt.plot(valid[['Close', 'Predictions']])

Stock Prices Prediction Using Machine Learning and Deep Learning (12)

Inference

The RMSE value is close to 105 but the results are not very promising (as you can gather from the plot). The predicted values are of the same range as the observed values in the train set (there is an increasing trend initially and then a slow decrease).

In the next section, we will look at two commonly used machine learning techniques – Linear Regression and kNN, and see how they perform on our stock market data.

Linear Regression

Introduction

The most basic machine learning algorithm that can be implemented on this data is linear regression. The linear regression model returns an equation that determines the relationship between the independent variables and the dependent variable.

Stock Prices Prediction Using Machine Learning and Deep Learning (13)

The equation for linear regression can be written as:

Here, x₁, x₂,….x_n represent the independent variables while the coefficients θ₁, θ₂, …. θ_n represent the weights. You can refer to the following article to study linear regression in more detail:

A comprehensive beginners guide for Linear, Ridge and Lasso Regression.

For our problem statement, we do not have a set of independent variables. We have only the dates instead. Let us use the date column to extract features like – day, month, year, mon/fri etc. and then fit a linear regression model.

Implementation

We will first sort the dataset in ascending order and then create a separate dataset so that any new feature created does not affect the original data.

#setting index as date valuesdf['Date'] = pd.to_datetime(df.Date,format='%Y-%m-%d')df.index = df['Date']#sortingdata = df.sort_index(ascending=True, axis=0)#creating a separate datasetnew_data = pd.DataFrame(index=range(0,len(df)),columns=['Date', 'Close'])for i in range(0,len(data)): new_data['Date'][i] = data['Date'][i] new_data['Close'][i] = data['Close'][i]

#create featuresfrom fastai.structured import add_datepartadd_datepart(new_data, 'Date')new_data.drop('Elapsed', axis=1, inplace=True) #elapsed will be the time stamp

This creates features such as:

‘Year’, ‘Month’, ‘Week’, ‘Day’, ‘Dayofweek’, ‘Dayofyear’, ‘Is_month_end’, ‘Is_month_start’, ‘Is_quarter_end’, ‘Is_quarter_start’, ‘Is_year_end’, and ‘Is_year_start’.

Note: I have used add_datepart from fastai library. If you do not have it installed, you can simply use the command pip install fastai. Otherwise, you can create these feature using simple for loops in python. I have shown an example below.

Apart from this, we can add our own set of features that we believe would be relevant for the predictions. For instance, my hypothesis is that the first and last days of the week could potentially affect the closing price of the stock far more than the other days. So I have created a feature that identifies whether a given day is Monday/Friday or Tuesday/Wednesday/Thursday. This can be done using the following lines of code:

new_data['mon_fri'] = 0for i in range(0,len(new_data)): if (new_data['Dayofweek'][i] == 0 or new_data['Dayofweek'][i] == 4): new_data['mon_fri'][i] = 1 else: new_data['mon_fri'][i] = 0

If the day of week is equal to 0 or 4, the column value will be 1, otherwise 0. Similarly, you can create multiple features. If you have some ideas for features that can be helpful in predicting stock price, please share in the comment section.

We will now split the data into train and validation sets to check the performance of the model.

#split into train and validationtrain = new_data[:987]valid = new_data[987:]x_train = train.drop('Close', axis=1)y_train = train['Close']x_valid = valid.drop('Close', axis=1)y_valid = valid['Close']#implement linear regressionfrom sklearn.linear_model import LinearRegressionmodel = LinearRegression()model.fit(x_train,y_train)

Results

#make predictions and find the rmsepreds = model.predict(x_valid)rms=np.sqrt(np.mean(np.power((np.array(y_valid)-np.array(preds)),2)))rms

121.16291596523156

The RMSE value is higher than the previous technique, which clearly shows that linear regression has performed poorly. Let’s look at the plot and understand why linear regression has not done well:

#plotvalid['Predictions'] = 0valid['Predictions'] = predsvalid.index = new_data[987:].indextrain.index = new_data[:987].indexplt.plot(train['Close'])plt.plot(valid[['Close', 'Predictions']])

Stock Prices Prediction Using Machine Learning and Deep Learning (14)

Inference

Linear regression is a simple technique and quite easy to interpret, but there are a few obvious disadvantages. One problem in using regression algorithms is that the model overfits to the date and month column. Instead of taking into account the previous values from the point of prediction, the model will consider the value from the same date a month ago, or the same date/month a year ago.

As seen from the plot above, for January 2016 and January 2017, there was a drop in the stock price. The model has predicted the same for January 2018. A linear regression technique can perform well for problems such as Big Mart sales where the independent features are useful for determining the target value.

k-Nearest Neighbours

Introduction

Another interesting ML algorithm that one can use here is kNN (k nearest neighbours). Based on the independent variables, kNN finds the similarity between new data points and old data points. Let me explain this with a simple example.

Consider the height and age for 11 people. On the basis of given features (‘Age’ and ‘Height’), the table can be represented in a graphical format as shown below:

Stock Prices Prediction Using Machine Learning and Deep Learning (15)

To determine the weight for ID #11, kNN considers the weight of the nearest neighbors of this ID. The weight of ID #11 is predicted to be the average of it’s neighbors. If we consider three neighbours (k=3) for now, the weight for ID#11 would be = (77+72+60)/3 = 69.66 kg.

Stock Prices Prediction Using Machine Learning and Deep Learning (16)

For a detailed understanding of kNN, you can refer to the following articles:

Introduction to k-Nearest Neighbors: Simplified
A Practical Introduction to K-Nearest Neighbors Algorithm for Regression

Implementation

#importing librariesfrom sklearn import neighborsfrom sklearn.model_selection import GridSearchCVfrom sklearn.preprocessing import MinMaxScalerscaler = MinMaxScaler(feature_range=(0, 1))

Using the same train and validation set from the last section:

#scaling datax_train_scaled = scaler.fit_transform(x_train)x_train = pd.DataFrame(x_train_scaled)x_valid_scaled = scaler.fit_transform(x_valid)x_valid = pd.DataFrame(x_valid_scaled)#using gridsearch to find the best parameterparams = {'n_neighbors':[2,3,4,5,6,7,8,9]}knn = neighbors.KNeighborsRegressor()model = GridSearchCV(knn, params, cv=5)#fit the model and make predictionsmodel.fit(x_train,y_train)preds = model.predict(x_valid)

Results

#rmserms=np.sqrt(np.mean(np.power((np.array(y_valid)-np.array(preds)),2)))rms

115.17086550026721

There is not a huge difference in the RMSE value, but a plot for the predicted and actual values should provide a more clear understanding.

#plotvalid['Predictions'] = 0valid['Predictions'] = predsplt.plot(valid[['Close', 'Predictions']])plt.plot(train['Close'])

Stock Prices Prediction Using Machine Learning and Deep Learning (17)

Inference

The RMSE value is almost similar to the linear regression model and the plot shows the same pattern. Like linear regression, kNN also identified a drop in January 2018 since that has been the pattern for the past years. We can safely say that regression algorithms have not performed well on this dataset.

Let’s go ahead and look at some time series forecasting techniques to find out how they perform when faced with this stock prices prediction challenge.

Auto ARIMA

Introduction

ARIMA is a very popular statistical method for time series forecasting. ARIMA models take into account the past values to predict the future values. There are three important parameters in ARIMA:

p (past values used for forecasting the next value)
q (past forecast errors used to predict the future values)
d (order of differencing)

Parameter tuning for ARIMA consumes a lot of time. So we will use auto ARIMA which automatically selects the best combination of (p,q,d) that provides the least error. To read more about how auto ARIMA works, refer to this article:

Build High Performance Time Series Models using Auto ARIMA

Implementation

from pyramid.arima import auto_arimadata = df.sort_index(ascending=True, axis=0)train = data[:987]valid = data[987:]training = train['Close']validation = valid['Close']model = auto_arima(training, start_p=1, start_q=1,max_p=3, max_q=3, m=12,start_P=0, seasonal=True,d=1, D=1, trace=True,error_action='ignore',suppress_warnings=True)model.fit(training)forecast = model.predict(n_periods=248)forecast = pd.DataFrame(forecast,index = valid.index,columns=['Prediction'])

Results

rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-np.array(forecast['Prediction'])),2)))rms

44.954584993246954

#plotplt.plot(train['Close'])plt.plot(valid['Close'])plt.plot(forecast['Prediction'])

Stock Prices Prediction Using Machine Learning and Deep Learning (18)

Inference

As we saw earlier, an auto ARIMA model uses past data to understand the pattern in the time series. Using these values, the model captured an increasing trend in the series. Although the predictions using this technique are far better than that of the previously implemented machine learning models, these predictions are still not close to the real values.

As its evident from the plot, the model has captured a trend in the series, but does not focus on the seasonal part. In the next section, we will implement a time series model that takes both trend and seasonality of a series into account.

Prophet

Introduction

There are a number of time series techniques that can be implemented on the stock prediction dataset, but most of these techniques require a lot of data preprocessing before fitting the model. Prophet, designed and pioneered by Facebook, is a time series forecasting library that requires no data preprocessing and is extremely simple to implement. The input for Prophet is a dataframe with two columns: date and target (ds and y).

Prophet tries to capture the seasonality in the past data and works well when the dataset is large. Here is an interesting article that explains Prophet in a simple and intuitive manner:

Generate Quick and Accurate Time Series Forecasts using Facebook’s Prophet.

Implementation

#importing prophetfrom fbprophet import Prophet#creating dataframenew_data = pd.DataFrame(index=range(0,len(df)),columns=['Date', 'Close'])for i in range(0,len(data)): new_data['Date'][i] = data['Date'][i] new_data['Close'][i] = data['Close'][i]new_data['Date'] = pd.to_datetime(new_data.Date,format='%Y-%m-%d')new_data.index = new_data['Date']#preparing datanew_data.rename(columns={'Close': 'y', 'Date': 'ds'}, inplace=True)#train and validationtrain = new_data[:987]valid = new_data[987:]#fit the modelmodel = Prophet()model.fit(train)#predictionsclose_prices = model.make_future_dataframe(periods=len(valid))forecast = model.predict(close_prices)

Results

#rmseforecast_valid = forecast['yhat'][987:]rms=np.sqrt(np.mean(np.power((np.array(valid['y'])-np.array(forecast_valid)),2)))rms

57.494461930575149

#plotvalid['Predictions'] = 0valid['Predictions'] = forecast_valid.valuesplt.plot(train['y'])plt.plot(valid[['y', 'Predictions']])

Stock Prices Prediction Using Machine Learning and Deep Learning (19)

Inference

Prophet (like most time series forecasting techniques) tries to capture the trend and seasonality from past data. This model usually performs well on time series datasets, but fails to live up to it’s reputation in this case.

As it turns out, stock prices do not have a particular trend or seasonality. It highly depends on what is currently going on in the market and thus the prices rise and fall. Hence forecasting techniques like ARIMA, SARIMA and Prophet would not show good results for this particular problem.

Let us go ahead and try another advanced technique – Long Short Term Memory (LSTM).

Long Short Term Memory (LSTM)

Introduction

LSTMs are widely used for sequence prediction problems and have proven to be extremely effective. The reason they work so well is because LSTM is able to store past information that is important, and forget the information that is not. LSTM has three gates:

The input gate: The input gate adds information to the cell state
The forget gate: It removes the information that is no longer required by the model
The output gate: Output Gate at LSTM selects the information to be shown as output

For a more detailed understanding of LSTM and its architecture, you can go through the below article:

Introduction to Long Short Term Memory

For now, let us implement LSTM as a black box and check it’s performance on our particular data.

Implementation

#importing required librariesfrom sklearn.preprocessing import MinMaxScalerfrom keras.models import Sequentialfrom keras.layers import Dense, Dropout, LSTM#creating dataframedata = df.sort_index(ascending=True, axis=0)new_data = pd.DataFrame(index=range(0,len(df)),columns=['Date', 'Close'])for i in range(0,len(data)): new_data['Date'][i] = data['Date'][i] new_data['Close'][i] = data['Close'][i]#setting indexnew_data.index = new_data.Datenew_data.drop('Date', axis=1, inplace=True)#creating train and test setsdataset = new_data.valuestrain = dataset[0:987,:]valid = dataset[987:,:]#converting dataset into x_train and y_trainscaler = MinMaxScaler(feature_range=(0, 1))scaled_data = scaler.fit_transform(dataset)x_train, y_train = [], []for i in range(60,len(train)): x_train.append(scaled_data[i-60:i,0]) y_train.append(scaled_data[i,0])x_train, y_train = np.array(x_train), np.array(y_train)x_train = np.reshape(x_train, (x_train.shape[0],x_train.shape[1],1))# create and fit the LSTM networkmodel = Sequential()model.add(LSTM(units=50, return_sequences=True, input_shape=(x_train.shape[1],1)))model.add(LSTM(units=50))model.add(Dense(1))model.compile(loss='mean_squared_error', optimizer='adam')model.fit(x_train, y_train, epochs=1, batch_size=1, verbose=2)#predicting 246 values, using past 60 from the train datainputs = new_data[len(new_data) - len(valid) - 60:].valuesinputs = inputs.reshape(-1,1)inputs = scaler.transform(inputs)X_test = []for i in range(60,inputs.shape[0]): X_test.append(inputs[i-60:i,0])X_test = np.array(X_test)X_test = np.reshape(X_test, (X_test.shape[0],X_test.shape[1],1))closing_price = model.predict(X_test)closing_price = scaler.inverse_transform(closing_price)

Results

rms=np.sqrt(np.mean(np.power((valid-closing_price),2)))rms

11.772259608962642

#for plottingtrain = new_data[:987]valid = new_data[987:]valid['Predictions'] = closing_priceplt.plot(train['Close'])plt.plot(valid[['Close','Predictions']])

Stock Prices Prediction Using Machine Learning and Deep Learning (20)

Inference

Wow! The LSTM model can be tuned for various parameters such as changing the number of LSTM layers, adding dropout value or increasing the number of epochs. But are the predictions from LSTM enough to identify whether the stock price will increase or decrease? Certainly not!

As I mentioned at the start of the article, stock price is affected by the news about the company and other factors like demonetization or merger/demerger of the companies. There are certain intangible factors as well which can often be impossible to predict beforehand.

Conclusion

Time series forecasting is a very intriguing field to work with, as I have realized during my time writing these articles. There is a perception in the community that it’s a complex field, and while there is a grain of truth in there, it’s not so difficult once you get the hang of the basic techniques.

Frequently Asked Questions

Q1. Is it possible to predict the stock market with Deep Learning?

A. Yes, it is possible to predict the stock market with Deep Learning algorithms such as moving average, linear regression, Auto ARIMA, LSTM, and more.

Q2. What can you use to predict stock prices in Deep Learning?

A. Moving average, linear regression, KNN (k-nearest neighbor), Auto ARIMA, and LSTM (Long Short Term Memory) are some of the most common Deep Learning algorithms used to predict stock prices.

Q3. What are the two methods to predict stock price?

A. Fundamental Analysis and Technical Analysis are the two ways of analyzing and predicting stock prices.

auto arimaKNNlinear regressionlive codingLSTMMoving averageprophetpythonStock market analysisStock predictionTime SeriesTime Series Forecasting

Aishwarya Singh02 May 2023

An avid reader and blogger who loves exploring the endless world of data science and artificial intelligence. Fascinated by the limitless applications of ML and AI; eager to learn and discover the depths of data science.

Deep LearningIntermediateMachine LearningProjectPython

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

James Verdant 25 Oct, 2018

Isn't the LSTM model using your "validation" data as part of its modeling to generate its predictions since it only goes back 60 days. Your other techniques are only using the "training" data and don't have the benefit of looking back 60 days from the target prediction day. Is this a fair comparison?

1231

Show 1 reply

Aishwarya Singh 26 Oct, 2018

Hi James,The idea isn't to compare the techniques but to see what works best for stock market predictions. Certainly for this problem LSTM works well, while for other problems, other techniques might perform better. We can add a lookback component with LSTM is an added advantage

123456

Jay 25 Oct, 2018

Getting index error ----------------------------------------------------------------------------IndexError Traceback (most recent call last)in1 #Results----> 2 rms=np.sqrt(np.mean(np.power((np.array(valid['Close']) - np.array(valid['Predictions'])),2)))3 rmsIndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

1232

Show 2 reply

Aishwarya Singh 26 Oct, 2018

Hi Jay,Please use the following command before calculating rmse valid['Predictions'] = 0valid['Predictions'] = closing_price. I have updated the same in the article

123456

BlackBirch 05 Jan, 2019

Thanks for sharing. I guess if you plot P(t) against P(t-1) you can pretty much get a chart similar to the LSTM results. If that is the case, then a simple bench mark for any of the models would be using yesterday's price as today's prediction. A model has to beat that, at least.

123456

Pankaj 26 Oct, 2018

After running the following codes-train['Date'].min(), train['Date'].max(), valid['Date'].min(), valid['Date'].max()(Timestamp('2013-10-08 00:00:00'),Timestamp('2017-10-06 00:00:00'),Timestamp('2017-10-09 00:00:00'),Timestamp('2018-10-08 00:00:00'))I am getting the following error :name 'Timestamp' is not definedPlease help.

1232

Show 2 reply

Aishwarya Singh 26 Oct, 2018

Hi Pankaj,The command is only train[‘Date’].min(), train[‘Date’].max(), valid[‘Date’].min(), valid[‘Date’].max() , the timestamp is the result I got by running the above command.

123456

Vishal 13 Nov, 2018

Pankaj use below codefrom pandas import Timestamp

123456

Zarief Marzuki Lao 26 Oct, 2018

Hi Aishwarya,Just curious. LSTM works just TOO well !!Is splitting dataset to train & valid step carry out after the normalizing step ?!i.e.#converting dataset into x_train and y_trainscaler = MinMaxScaler(feature_range=(0, 1))scaled_data = scaler.fit_transform(dataset)thendataset = new_datatrain = dataset[:987]valid = dataset[987:]Then thisx_train, y_train = [], []for i in range(60,len(train )): # <- replace dataset with train ?!x_train.append(scaled_data[i-60:i,0])y_train.append(scaled_data[i,0])x_train, y_train = np.array(x_train), np.array(y_train)Guide me on this.Thanks

1231

Show 1 reply

Aishwarya Singh 26 Oct, 2018

Hi Zarief,Yes, the train and test set are created after scaling the data using the for loop :x_train, y_train = [], []for i in range(60,len(train )):x_train.append(scaled_data[i-60:i,0]) #we have used the scaled data herey_train.append(scaled_data[i,0])x_train, y_train = np.array(x_train), np.array(y_train)Secondly, the command dataset = new_data.values will be before scaling the data, as shown in the article, since dataset is used for scaling and hence must be defined before.

123456

rohit 26 Oct, 2018

hiI'm getting below error...''">>> #import packages>>> import pandas as pdTraceback (most recent call last):File "", line 1, inimport pandas as pdImportError: No module named pandas""""

1232

Show 2 reply

Aishwarya Singh 29 Oct, 2018

Hi rohit,Is the issue resolved? Have you worked with pandas previously?

123456

Nick 18 Jan, 2019

Try runningpip install pandasin a command line

123456

Shan 27 Oct, 2018

Hi..Thanks for nicely elaborating LSTM implementation in the article.However, in LSTM rms part if you can guide, as I am getting the following error :valid['Predictions'] = 0.0valid['Predictions'] = closing_pricerms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-np.array(valid['Predictions'])),2)))rms##############################################################################IndexError Traceback (most recent call last)in ()----> 1 valid['Predictions'] = 0.02 valid['Predictions'] = closing_price3 rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-np.array(valid['Predictions'])),2)))4 rmsIndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

123

Jay 27 Oct, 2018

Still same error ----------------------------------------------------------------------------IndexError Traceback (most recent call last)in----> 1 valid['Predictions'] = closing_priceIndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indicesAnd also why are we doing valid['Predictions'] = 0 and valid['Predictions'] = closing_price instead of valid['Predictions'] = closing_price

1231

Show 1 reply

Aishwarya Singh 27 Oct, 2018

Yes you can skip the line but it will still show an error because the index hasn't been defined. Have you followed the code from the start? Please add the following code lines and check if it worksnew_data.index = data['Date'] #considering the date column has been set to datetime formatvalid.index = new_data[987:].indextrain.index = new_data[:987].indexLet me know if this works. Other wise share the notebook you are working on and I will look into it.

123456

Roberto 27 Oct, 2018

Hi,Nice article.I have installed fastai but I am getting the following error:ModuleNotFoundError: No module named 'fastai.structured'Any idea?

1231

Show 1 reply

Aishwarya Singh 31 Oct, 2018

Hi Roberto,Directly clone it from here : https://github.com/fastai/fastai . Let me know if you still face an issue.

123456

Jingmiao 29 Oct, 2018

Hello AISHWARYA,I dont know what's your motivation to spend such a long time to write this blogBut thank you soooooo much!!!Appreciate your time for both the words and codes (easy to follow) !!!Such a great work!!!

1231

Show 1 reply

Aishwarya Singh 29 Oct, 2018

Really glad you liked the article. Thank you!

123456

Vedamurthy KB 31 Oct, 2018

new_data.index=data['Date']valid.index=new_data[987:].indextrain.index=new_data[:987].indexgivesAttributeError Traceback (most recent call last)in ()1 new_data.index=data['Date']----> 2 valid.index=new_data[987:].index3 train.index=new_data[:987].indexAttributeError: 'numpy.ndarray' object has no attribute 'index'valid['Predictions'] = 0valid['Predictions'] = closing_pricegivesIndexError Traceback (most recent call last)in ()----> 1 valid['Predictions'] = 02 valid['Predictions'] = closing_priceIndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indicesV good article. I am getting above errors. Kindly help solving it

1231

Show 1 reply

Aishwarya Singh 31 Oct, 2018

Hi,Please print the validation head and see if the index values are actually the dates or just numbers. If they are numbers, change the index to dates. Please share the screenshot here or via mail (dropped a mail)

123456

Ravi Shankar 31 Oct, 2018

Hi,Thanks for putting the efforts in writing the article.If you believe LSTM model works this well, try buying few shares of Tata Global beverages and let us know the returns on the same. I guess, you would understand the concept of over-fit.Thanks,Ravi

1231

Show 1 reply

Aishwarya Singh 31 Oct, 2018

Hi Ravi,I actually did finally train my model on the complete data and predicted for next 10 days (and checked against the results for the week). The first 2 predictions weren't exactly good but next 3 were (didn't check the remaining). Secondly, I agree that machine learning models aren't the only thing one can trust, years of experience & awareness about what's happening in the market can beat any ml/dl model when it comes to stock predictions. I wanted to explore this domain and I have learnt more while working on this dataset than I did while writing my previous articles.

123456

Moobi 11 Nov, 2018

Its nice tutorial, thanks. I need to know how can i predict just tomorrow’s price?

1231

Show 1 reply

Aishwarya Singh 12 Nov, 2018

Hi,To make predictions only for the next day, the validation set should have only 1 row (with past 60 values).

123456

Pabitra 12 Nov, 2018

What is the difference between last and closing price?

1231

Show 1 reply

Aishwarya Singh 13 Nov, 2018

The difference is not significant. I plotted the two variables and they overlapped each other.

123456

Miguel 13 Nov, 2018

Hi, thanks for the article.I got this error:rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))ValueError: operands could not be broadcast together with shapes (1076,) (248,)Can you help me on this?

1232

Show 2 reply

Aishwarya Singh 13 Nov, 2018

Hi,Looks like the validation set and predictions have different lengths. Can you please share your notebook with me? I have sent you a mail on the email ID you provided.

123456

Isaac 30 Nov, 2018

I ran into the same error as ValueError: operands could not be broadcast together with shapes (1088,) (248,). Guidance towards resolution would be appreciated.

123456

Alex 24 Nov, 2018

Thank you so much for the code, you inspire me a lot. I applied your algorithm for my example and it it works fine. Now i have to make it predict the price for the next 5 years, do you know how to achieve that? I should use all the data without splitting into test and train for training and somehow generate new dates in that array, then predict the value for them. I tried to modify your code but i couldn't figure it out. I'm new to ML and it's really hard to understand those functions and classes.Thank you very much in advance

1231

Show 1 reply

Aishwarya Singh 05 Dec, 2018

Hi Alex,Currently the model is trained to look at the recent past data and make predictions for the next day. To make predictions for 5 years in future, we'll first have to change the way we train model here, and we'll need a, much bigger dataset as well.

123456

Shabbir Anaswala 05 Dec, 2018

Hi,I am getting this errorx_train_scaled = scaler.fit_transform(x_train)Traceback (most recent call last):File "", line 1, inx_train_scaled = scaler.fit_transform(x_train)File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\base.py", line 517, in fit_transformreturn self.fit(X, **fit_params).transform(X)File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py", line 308, in fitreturn self.partial_fit(X, y)File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py", line 334, in partial_fitestimator=self, dtype=FLOAT_DTYPES)File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\validation.py", line 433, in check_arrayarray = np.array(array, dtype=dtype, order=order, copy=copy)TypeError: float() argument must be a string or a number, not 'Timestamp'

Hello,I have questions about your model LSTM. I don't understand the building of "X_test".I have understood the aim and the bulding of your variable "inputs", but I believe that your variable "X_test" use the observed values.Imagine that the dataset has 500 values and we use the first 400 for the train and you want to predict the others.For the first day, you will use the values 340 until 400 and you can predict the value 401. Ok no problem !For the second day, you will use the values 341 until 401. If you want to predict on several months, you have to use the predicted value 401. But your value 401 is the value contained in the list "inputs". So you don't use the predicted variable but you use the observed value.Actually if your model can predict on 2 months, I can replace values in valid and the curve of predictions can't change (because I don't modify the train). I tried to replace all the observed values between 401 and 500 by 0... The predicted values change completely !I feel that you do predictions using the real value of the day before to compute the actual day but you don't predict the stock market on several months.My question is : what part didn't I understand ?I thank you and have a good day :)

123

@run 27 Jan, 2019

Hey Aishwarya, your article is super helpful. Thanks a ton. Keep going!!!

1231

Show 1 reply

Aishwarya Singh 28 Jan, 2019

Thanks Arun!

123456

Frank 28 Jan, 2019

Thanks a lot for sharing this. I keep wondering if we just use the previous close as a prediction for next day, would the chart essentially the same as the LTMS? Can LTMS beat that simple prediction?

123

deep 28 Jan, 2019

Hey Aishwarya, a related question. So say if you need to build a system based on LSTM to predict 50 stocks on any day. Would you create 50 LSTM models and select the right stock model at runtime OR train one LSTM model only for all 50 stocks ?? What would be your approach ?

123

antoine73 29 Jan, 2019

Hello,I do not understand the construction of forecasts in the LSTM method. I note N the first value to predict. The objective is to compare the predicted values N to N + 10 with the observed values. To predict the value N, you take the 60 previous observed values ok. To predict the value N + 1, what values do you take?I don't understand if you use :- 59 previous observed values without taking the value in N- 59 previous values + the predictive value in N- 59 previous values + the observed value in NI try to understand the methodology but this little point is blocking me.All the rest of the article is very clear.I thank you :)

123

Doaa 29 Jan, 2019

Hi Aishwarya,please can you tell me about books explain how to use LSTM in stock market in detailsthanks in advance

123

MikeMikeMike 30 Jan, 2019

Aishwaryai, I found your article very interesting. I'm not familiar with programing but I was able to follow the reason for each step in the process.I do have a question.You noted that there are many other factors that will ultimately affect the market. As someone who works in marketing and has to be plugged into social media I use many tools that tell me how a company is being perceived. Twitter and Facebook both offer lot of opportunity for social listening and the tools we use in advertising/marketing to measure take the temperature of the market are quite powerful.Would it be possible to incorporate some machine learning to find patterns in the positive/negative sentiment measurements that are constantly active to help predict when stock values are going to change and in which direction? I feel like your article is using current and past market data but doesn't incorporate social perception.http://www.aclweb.org/anthology/W18-3102The study linked above only focuses on stock related posts made from verified accounts but didn't take public perception measurements into account in their data. With how connected to social media the world has become I wonder if combining the data you used in your article with data from social listening tools used by marketers could create predictive tools that would essentially game the system.

1231

Show 1 reply

smit 09 Mar, 2019

u can use sentimental analysis.....rest u can search yourself.......u can watch sirajvideo....for more info

123456

Rahul Saini 01 Feb, 2019

Hi AISHWARYA SINGH,You blog is very impressive.Could you please help me for building a system with following requirements:The system will do trade training using huge data (historical/past trades and stocks prices) and incremental data from new daily trades and prices.The system will include a user-friendly interface which means that everyone even people with low knowledge with computers will be able to use this system on any computer/laptop.The user will have the option to run a prediction any time he wants, the user will be able to make the system to do trade training any time he wants.As mentioned above the system will be able to do training based on historical/past data and incremental new daily data, which means data will be updated consistently so the system will be always updated.Thanks in advance

123

Slavko 02 Feb, 2019

# Complete code with yahoo data reader# Calculated dataset length trainig set length and prediction length (plen)# coding: utf-8# In[1]:#import packagesfrom pandas_datareader import dataimport fix_yahoo_finance as yfyf.pdr_override() # <== that's all it takes :-)import pandas as pdimport numpy as np#to plot within notebookimport matplotlib.pyplot as pltget_ipython().run_line_magic('matplotlib', 'inline')#setting figure sizefrom matplotlib.pylab import rcParamsrcParams['figure.figsize'] = 20,10#for normalizing datafrom sklearn.preprocessing import MinMaxScalerscaler = MinMaxScaler(feature_range=(0, 1))#read the file#df = pd.read_csv('NSE-BSE.csv')#df['Date'] = df.Date.apply(# lambda x: pd.to_datetime(x).strftime('%Y-%m-%d'))# Define the instruments to download. We would like to see Apple, Microsoft and the S&P500 index.#tickers = 'PYPL'tickers = 'AMD'# We would like all available data from 01/01/2000 until 12/31/2016.start_date = '2015-01-01'end_date = '2019-02-01'# download dataframedf = data.get_data_yahoo(tickers, start=start_date, end=end_date)df.columns = [c.replace(' ', '_') for c in df.columns]#print the taildf.tail()#df.dtypes# In[2]:#plotplt.figure(figsize=(16,8))plt.plot(df['Close'], label='Close Price history')# In[3]:#importing required librariesfrom sklearn.preprocessing import MinMaxScalerfrom keras.models import Sequentialfrom keras.layers import Dense, Dropout, LSTM#creating dataframedata = df.sort_index(ascending=True, axis=0)new_data = pd.DataFrame(index=range(0,len(df)),columns=['Date', 'Close'])for i in range(0,len(data)):new_data['Date'][i] = pd.to_datetime(df.index[i], unit='D').date()new_data['Close'][i] = data['Close'][i]#setting indexnew_data.index = new_data.Datenew_data.drop('Date', axis=1, inplace=True)#Predict number of daysplen = 80#creating train and test setsdataset = new_data.valuestrain = dataset[0:len(data)-plen,:]valid = dataset[len(data)-plen:,:]# Display data set lennew_data.shape, train.shape, valid.shape# In[4]:#converting dataset into x_train and y_trainscaler = MinMaxScaler(feature_range=(0, 1))scaled_data = scaler.fit_transform(dataset)x_train, y_train = [], []for i in range(plen,len(train)):x_train.append(scaled_data[i-plen:i,0])y_train.append(scaled_data[i,0])x_train, y_train = np.array(x_train), np.array(y_train)x_train = np.reshape(x_train, (x_train.shape[0],x_train.shape[1],1))# In[5]:# create and fit the LSTM networkmodel = Sequential()model.add(LSTM(units=50, return_sequences=True, input_shape=(x_train.shape[1],1)))model.add(LSTM(units=50))model.add(Dense(1))model.compile(loss='mean_squared_error', optimizer='adam')model.fit(x_train, y_train, epochs=1, batch_size=1, verbose=2)# In[6]:#predicting values, using past plen from the train datainputs = new_data[len(new_data) - len(valid) - plen:].valuesinputs = inputs.reshape(-1,1)inputs = scaler.transform(inputs)X_test = []for i in range(plen,inputs.shape[0]):X_test.append(inputs[i-plen:i,0])X_test = np.array(X_test)X_test = np.reshape(X_test, (X_test.shape[0],X_test.shape[1],1))closing_price = model.predict(X_test)closing_price = scaler.inverse_transform(closing_price)# In[7]:rms=np.sqrt(np.mean(np.power((valid-closing_price),2)))rms# In[9]:#for plottingtrain = new_data[:len(data)-plen]valid = new_data[len(data)-plen:]valid['Predictions'] = 0valid['Predictions'] = closing_pricedts=train['Close']dtv=valid[['Close','Predictions']]plt.figure(figsize=(16,8))plt.grid()plt.plot(dts[plen:])plt.plot(dtv[:plen])dtv.tail()

123

Harmeet 03 Feb, 2019

Hi there, thanks for the detailed explanation.Just a simple question..The data set had many features but we chose 'Date' feature to work with. Why didn't we consider other features? What is the approach that we take while choosing the feature set?And, what the need to separate Date feature to do linear regression?Thanks

123

Pratik 04 Feb, 2019

HI Aishwarya my name is pratik am a research engineer i have done similar work not only with stock data but also desigjn sentiment analyzer to get the sentiment of the news as well as the twitter data to get the cumilative score for final prediction i have use cnn and random forest i think ur normalization need to be modified there use EWMA panda techinique to smoothen the results and it will enhance ur prediction i think we have to use reinforcement technique as a reward for dip in prioce and high in price due to news or various factors this will enhace the results i didnt try yet but i feel reinforement is the solution for stock price problem

123

Alok 07 Feb, 2019

Hi, Aishwarya I am working on research on the same topic and I have a strong interest in this. SO, could you please send me the full working of this article along with the code. My email id is [emailprotected]

123

Archit Jain 07 Feb, 2019

help me to solve this errorrms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))ValueError: operands could not be broadcast together with shapes (0,) (248,)

123

Sam 09 Feb, 2019

Hi Aishwarya,could you share the full code please.I am getting the below error:ValueError Traceback (most recent call last)in ()1 #calculate rmse----> 2 rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))ValueError: operands could not be broadcast together with shapes (0,) (248,)1

123

Muralidharan 11 Feb, 2019

HiHow I can solve this error?AttributeError: 'DataFrame' object has no attribute 'Date'Thanks

123

Muralidharan 11 Feb, 2019

HiAfter dropping the date column(Linear Regression method), I am getting an error like this.AttributeError: 'DataFrame' object has no attribute 'Date'How do I resolve this error?

123

Muralidharan 11 Feb, 2019

HiCould please share the entire working code.Thanks

123

Haroon Balwa 11 Feb, 2019

Thank you for this great blog! I have a question of how to code the following issues in the given dataset of data_stock.csv1. Which stocks are apparently similar in performance2. Identify which all stocks are moving together and which all stocks are diff from each other?3. How many unique patterns that exist in the historical stock data set, based onn fluctuations in price.

123

Alex 18 Feb, 2019

Make no mistake, at the expense of LSTM. The model simply takes the data from the previous days and follow trends. This model can't even predict up or down the stock will go. It's easy to check with the following code-the prediction accuracy is only 47%.valid['R_Close'] = valid['Close'].diff()/valid['Close'].shift(1)valid['R_Pred'] = valid['Predictions'].diff()/valid['Predictions'].shift(1)valid['event'] = valid['R_Close']/valid['R_Pred']print(len(valid[valid['event'] > 0])/len(valid))plt.plot(valid[['R_Close','R_Pred']])

123

Tanmay Jain 21 Feb, 2019

---------------------------------------------------------------------------i am getting this error please help me to debug the error.ValueError Traceback (most recent call last)in ()34 #calculate rmse----> 5 rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))6 rmsValueError: operands could not be broadcast together with shapes (0,) (248,)

123

srujith 22 Feb, 2019

Hi, thanks for the article.I got this error:rms=np.sqrt(np.mean(np.power((np.array(valid[‘Close’])-preds),2)))ValueError: operands could not be broadcast together with shapes (1076,) (248,)Can you help me on this?

123

srujith 22 Feb, 2019

Hi,i got this error can you please help me through this#calculate rmserms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))print (rms)ValueError Traceback (most recent call last)in ()1 #calculate rmse----> 2 rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))3 print (rms)4ValueError: operands could not be broadcast together with shapes (410,) (248,)

123

MorfiMen 25 Feb, 2019

How can I predict the future 30 values? thank you so much

123

Alex 25 Feb, 2019

You should consider implementing the simplest model you can think of. Since your LSTM uses all data up to a given date to predict just the next date, a good benchmark model would be the following: a model, that simply repeats the value from the last day to predict the value for the following day. In pandas this is simply implemented as:prediction = new_data.shift(1)To get a plot of that model:new_data.join(prediction, lsuffix='_train', rsuffix='_predict').plot())With that simple model I get a rms of 11.85 (not much worse compared to the 11.77 from LSTM).

123

Mukesh Kumar 28 Feb, 2019

Hi,I think you should only use train data to fit MinMaxScaler and then transform train and valid data separately.Thanks

123

Varun Singh 02 Mar, 2019

HeyI tried working on this code and i am getting an error, "float() argument must be a string or a number, not 'Timestamp' "Can you please help me solve it

123

John 03 Mar, 2019

Hi Aishwarya,Thanks very much for taking the time to post this article! I have been looking for an intro into time series analysis of stock prices. Your information was a great start and I was able to duplicate your LSTM code output. I apologize for the naive question on Python coding but, regarding your LSTM method to predict the next day closing price based on prior 60 day close prices, how would you output the next day close price prediction? For example, if I print the last day LSTM results from your dataset, I see:Date Actual Close Predicted Close2019-01-04 213.8 228.571320If it were real time (I mean if the current day was actually Jan 4th), how would you output the predicted next day close for Jan 5th?Thanks again...nice article!

123

Alok 04 Mar, 2019

Hi, Aishwarya could you please tell me that for measurement of the performance of the algorithms what metric are u taking?

123

Joseph 04 Mar, 2019

The error shows up while calculating rms#calculate rmserms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))rms---------------------------------------------------------------------------ValueError Traceback (most recent call last)in1 #calculate rmse----> 2 rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))3 rmsValueError: operands could not be broadcast together with shapes (0,) (248,)When edited according to AISHWARYA SINGH who says,Please use the following command before calculating rmse:valid['Predictions'] = 0valid['Predictions'] = closing_priceAfter editing the code like this,#calculate rmsevalid['Predictions'] = 0valid['Predictions'] = closing_pricerms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))rms---------------------------------------------------------------------------NameError Traceback (most recent call last)in1 #calculate rmse2 valid['Predictions'] = 0----> 3 valid['Predictions'] = closing_price4 rms=np.sqrt(np.mean(np.power((np.array(valid['Close'])-preds),2)))5 rmsNameError: name 'closing_price' is not definedBasically, there's error in both ways. Is it because of that I'm using Python 3? Help would be appreciated!

123

Marcelo 05 Mar, 2019

Hi Asihwarya,Thanks for sharing this very interesting tutorial. Can you please share the full working code?Regards,Marcelo

123

yuganshu sanjay tickoo 08 Mar, 2019

mam can you help me in how to find the accuracy of the model

123

Haoran 12 Mar, 2019

Hi Aishwarya,could you please share the data with me, Thank you.

123

Logesh 13 Mar, 2019

lets assume we have 1000 instances of data separated into training and validation. Can we predict and get the value of the 1005th instance that is not present in the data?

123

ml-learner1641 19 Mar, 2019

Is there any way to do this with tensor-flow or anything other than keras? If so, can you point me in the right direction? I'm running into problems downloading keras and I haven't found a solution yet. Thanks Aishwarya!

123

CarreyWong 21 Mar, 2019

In section LSTMI get this error, could u please tell me why and how to fix this when you are free? thank u very much.TypeError Traceback (most recent call last)in ()55 X_test = np.reshape(X_test, (X_test.shape[0],X_test.shape[1],1))56 print(X_test)---> 57 closing_price = model.predict(X_test)58 closing_price = scaler.inverse_transform(closing_price)59TypeError: data type not understood

123

Justin 22 Mar, 2019

Hi,Thanks for the article. Regarding the following line: -#predicting 246 values, using past 60 from the train datainputs = new_data[len(new_data) - len(valid) - 60:].valuesMay I know what value obtained for : -len(new_data) ?len(valid) ?inputs ?

1231

Show 1 reply

Aishwarya Singh 27 Mar, 2019

Hi Justin,Valid and closing price length is 248, input is 308, and new data 1235 same as the len of df

123456

Tobias Wang 24 Mar, 2019

Hi Aishwarya,I was suspicious of your program, since it worked it little too well, so I played around with the program, and it after I tried to make it predict some future stocks (by making the validation set go to the future, and filling the rows with zeroes ), and you can imagine my surprise when the prediction said the stocks would drop like a stone. I investigated further, and found that your data leaks in the LSTM implementation, and that's why it works so well.

1231

Show 1 reply

Aishwarya Singh 25 Mar, 2019

Hi,Don't fill the rows with zero! fill it with the predicted value. So you made a prediction for next day, use that to predict the third day. Even if you do not use the validation set as done here, use the predictions by your model. Giving it zero as input for the last 2-3 days, the model would understand that yesterday's closing price was zero, and will show a drastic drop.

123456

Justin 26 Mar, 2019

Hi Aishwarya,#predicting 246 values, using past 60 from the train datainputs = new_data[len(new_data) - len(valid) - 60:].valuesRegarding the above line, may I know what are the values obtained for the following:len(new_data) ?len(valid) ?inputs ?Thanks.

123

Diego 27 Mar, 2019

Hi! Great post! If I want to predict 10 or 15 days instead of 1. Which part of the code I have to change? I can´t get it..thanks!!!!!!!!

123

Diego Fernandez 27 Mar, 2019

Hi! Great post! I can´t figure how to predict more tan one day. Could you help me? which line do you have to change?greetings!!

123

varna 28 Mar, 2019

I'm using jupyter notebook in anaconda3 navigator and I'm getting this errorModuleNotFoundError: No module named 'fastai'. can you help me to resolve this

123

Gaurav 29 Mar, 2019

Hi Aishwarya,I am working on product development for financial domain. If you could share code project with me and along with some material which can help me learning ML basics that would be great. My email id is gcchavda[a]gmail[dt]com.

123

K N MUZAMIL 30 Mar, 2019

i want data on which you performed above exercises as i am learning data science course i need to perform projects.over data.....

123

Rajesh 30 Mar, 2019

Hello,It was a great experience learning from your article, it would have been much better if you explained each line of the code in all the approaches.My doubt is why is the RMSE value varying on each execution on the same data if I execute immediately in a short period of time and secondly correct me if I am wrong the orange line in the predicted graph for each model has to coincide with the original blue line only then we could conclude it to be the right forecast?please reply ASAP

123

Prashant 02 Apr, 2019

Hi AishwaryaHelp me to print the predicted values along with plotting of values.and secondly how can we implement the same code for intraday values when the date-time values are like4/2/2019 2:07:00 AMthirdly what changes are required to see next 5 days values and print themThanks for the code, it has helped me to start with it.

123

Marcel Delhez 02 Apr, 2019

Fully agree with Tobias his analysis.By the way, I like the article but not the way LSTM is implemented. Yes, it uses data it is not supposed to know if you want to compare different types of models. And for you information, but here goal is obviously just to compare models, machine learning cannot predict financial markets just based on historical prices.would be too easy.But, as I wrote, article present different models and that is valuable for many people as I see.

123

Metus 02 Apr, 2019

Hi Aishwarya,I want you to explain all your code about LSTM.but now i want to know why you use unit = 50 in this below line'model.add(LSTM(units=50, return_sequences=True, input_shape=(x_train.shape[1],1)))'Thank you

123

J S Abhishek 05 Apr, 2019

Hi Aishwarya,Really an intersting article,Please check if the following is correct. To predict the future values of more than 5 years, I've used the following code to predict next 100 days:pred_price=[]num_pred=100X_test = np.reshape(X_test, (X_test.shape[0],X_test.shape[1],1))print(X_test.shape)print(X_test[0,:,0])X_test = np.array(X_test)for i in range(num_pred):closing_price = model.predict(X_test)print(closing_price)X_test=np.delete(X_test[0,:,0],0)#print(X_test.shape)print(X_test)X_test=np.append(X_test,closing_price)#print(X_test.shape)print(X_test)X_test=np.reshape(X_test,(1,X_test.shape[0],1))print(X_test.shape)closing_price = scaler.inverse_transform(closing_price)pred_price.append(closing_price)pred_price=np.array(pred_price)print(type(pred_price))print(pred_price.shape)pred_price=np.reshape(pred_price,(num_pred,1))print(pred_price.shape)print(pred_price)

123

J S Abhishek 05 Apr, 2019

Hi,Firstly, really an amazing article written by you on AnalyticsVidhya. Thanks a lot for the post.So, I tried to predict more than test_set. The code goes as follows:pred_price=[]num_pred=10X_test = np.reshape(X_test, (X_test.shape[0],X_test.shape[1],1))print(X_test.shape)print(X_test[0,:,0])X_test = np.array(X_test)for i in range(num_pred):closing_price = model.predict(X_test)print(closing_price)X_test=np.delete(X_test[0,:,0],0)#print(X_test.shape)print(X_test)X_test=np.append(X_test,closing_price)#print(X_test.shape)print(X_test)X_test=np.reshape(X_test,(1,X_test.shape[0],1))print(X_test.shape)closing_price = scaler.inverse_transform(closing_price)pred_price.append(closing_price)pred_price=np.array(pred_price)print(type(pred_price))print(pred_price.shape)pred_price=np.reshape(pred_price,(num_pred,1))print(pred_price.shape)print(pred_price)This has shown that the values are always increasing. I've attached the python workbook file and data csv file. Is there anything wrong with the code. Waiting for your response.

123

Rishab Kumar 06 Apr, 2019

I am having an error: cannot import name 'blosc_version'Please help me.

123

Ping Xu 07 Apr, 2019

Thank you for this great work. The prediction results were only 63 rows not like said 248 as the same scripts were use as you posted. Could you please help? Thanks so much.Also I am using Python not Anconda. I could not install pystan to verify the Prophet which needs FBProphet that needs pystan. Any comments? Do I have to use Anconda? When I installed Anconda which supposed have pandas, numpy but error messages were it could not find them, not could it find the access to install. Thanks again.

123

Arjun 07 Apr, 2019

Hi,My name is Arjun, i like to do project using ML. I like to design a prediction model to predict price of spices.Can I use this same code to develop a model?

123

Impana S 10 Apr, 2019

If I could get 'Prediction' value for single date how to write code. I tried for newdata.index but couldn't get date for prediction

123

Omar El Omeiri 12 Apr, 2019

I'm not sure why this happened.the second last line of code:closing_price = model.predict(X_test)apparently goes in an infinite loop, hours running but no output.I have a pretty new Macbook Pro, processing power is not an issue.is this normal?

123

Dev Thakkar 14 Apr, 2019

Hi Aishwarya,I want to use close price and volume both for training the model. I also want to predict prices for next 3 months. Can you please help me with this as I am new to python.

123

James M. Yunker (please do not post, call me anonymous) 15 Apr, 2019

Please send to me the working python code. Also, please send the workaround I do not have fastai installed.

123

Sanheen Sethi 16 Apr, 2019

I train the data , and see the predictions of test data , and it also worked on bitcoin data i used , but How can i predict by giving new values means like i want to predict for 17 apr 2019 , which line of statement can do this ?

123

Mazhar 01 May, 2019

In KNN it is giving me an error : invalid literal for float()can you please help me with that

123

Sudeep Dasgupta 03 May, 2019

How you are going to predict future stock price, here you are using the test_X and test_y from the original datasets. that is the reason it is giving accurate prediction using LSTM when you are going to predict future stock price and need to create test_X and test_y and feed the value to predict it wont give good result.

123

Peter 11 May, 2019

Hello Aishwarya,Thanks for sharing! If I may ask.1) Why do you make the cut-off at 987 for training? Why not 988 or any other number? Any rule of thumb to go by here?2) Why look at 60 days to predict 61st? Would looking at more days back yield a "better" prediction? Any rule of thumb for this?3) Why set units to 50? As above, is there a rule of thumb?4) Could you please show me a quick example with what you mean by "You need to create a validation set with only 10 rows as input to the LSTM model." when trying to predict more than 1 day in advance? I cant really see where to make the change(s) to make that work.Many thanks!Peter

123

Imran 12 May, 2019

Hi Aish,Thanks for this great article.I tried the but Im getting errors in fastai and other snippets. Can you please send me the code?

123

Ali Raza 12 May, 2019

Aishwarya I appreciate your work, I am engineering student and working on Machine Learning, I was trying to implement that code to my own but I found some error, from the LinearRegression (fastai library) It does not work, when I install the library fastai, it shows the error [no module torch] and I can find torch on internet I found pytorch which is not working for that. Please help me...

123

rekha 17 May, 2019

hI,Which are the other sequence prediction time series problems where LSTM can be applied

123

Frank 21 May, 2019

Hi, The average % error from LSTM model results is around 4-5% per day. If I use previous day close as a prediction for today's close, I got a better prediction with average % error of 2%. The graph will look even prettier than LSTM if we use previous close as a prediction. But we all know using previous day close as a prediction won't make you money.

123

Pau 27 May, 2019

Hi, I tried out this algorithm with real data from Binance. It works well, but I'm afraid it does not help predict anything. let's see whether I am missing anything (and I hope so!).I don't want to have any test set because I want to guess the results in the real world, so I trained everything. When predicting, in this case, we have 2 axes x: dates, y: price.We should be able to give the trained model an x coordinate (time units) and then it should return the y (prices) it estimates will be the real one.But after looking into it, I realised prices are being used when predicting instead of dates. Why? In the real world, I don't have these prices! I have only time units! I would like to use these time units in order to predict which prices will be in each one of them, and not using prices I won't have.Am I missing anything? And please, that's what I wanna hear!

123

Tyler 28 May, 2019

Hello,I'm having trouble getting past the fastai section. I can't download the package. Each time I try , I get a memory error (i don't think my server can handle the entire package download...it's almost a GB). I was trying to recreate it like you mentioned using simple loops, and I was able to get something. Here is the code:for i in range(0,len(new_data)):new_data['Year'][i] = new_data['Date'][i].yearnew_data['Month'][i] = new_data['Date'][i].monthnew_data['Week'][i] = new_data['Date'][i].strftime("%U")new_data['Day'][i] = new_data['Date'][i].daynew_data['Dayofweek'][i] = new_data['Date'][i].strftime("%w")new_data['Dayofyear'][i] = new_data['Date'][i].strftime("%j")if new_data['Date'][i].day == 1:new_data['Is_month_start'][i] = 1tomorrow = new_data['Date'][i]+timedelta(days=1)if tomorrow.day == 1:new_data['Is_month_end'][i] = 1if new_data['Date'][i].strftime("%d %m") in q_end:new_data['Is_quarter_end'][i] = 1if new_data['Date'][i].strftime("%d %m") in q_start:new_data['Is_quarter_start'][i] = 1if new_data['Date'][i].strftime("%d %m") == '01 01':new_data['Is_year_start'][i] = 1if new_data['Date'][i].strftime("%d %m") == '31 12':new_data['Is_year_end'][i] = 1But when I get to the next section I'm having trouble with the LRM. Can you confirm that this adds the proper columns and what not?Many thanks, and thanks for a great guide.

123

Tyler Tresslar 28 May, 2019

Hey,Thanks, this is a really great article. I'm having some issues in the linear regression part. I was not able to download the fastai package (memory error..) so I tried to go at it with simple loops instead. Here is my code:for i in range(0,len(new_data)):new_data['Year'][i] = new_data['Date'][i].yearnew_data['Month'][i] = new_data['Date'][i].monthnew_data['Week'][i] = new_data['Date'][i].strftime("%U")new_data['Day'][i] = new_data['Date'][i].daynew_data['Dayofweek'][i] = new_data['Date'][i].strftime("%w")new_data['Dayofyear'][i] = new_data['Date'][i].strftime("%j")if new_data['Date'][i].day == 1:new_data['Is_month_start'][i] = 1tomorrow = new_data['Date'][i]+timedelta(days=1)if tomorrow.day == 1:new_data['Is_month_end'][i] = 1if new_data['Date'][i].strftime("%d %m") in q_end:new_data['Is_quarter_end'][i] = 1if new_data['Date'][i].strftime("%d %m") in q_start:new_data['Is_quarter_start'][i] = 1if new_data['Date'][i].strftime("%d %m") == '01 01':new_data['Is_year_start'][i] = 1if new_data['Date'][i].strftime("%d %m") == '31 12':new_data['Is_year_end'][i] = 1new_data.head(35)I'm not sure if I made an error here or not, but everything seems to come out ok. I don't know exactly what the output should look like though. Maybe you could include a new_data.head() in your code, just to have an idea of what the data looks like?Anyway, when I try to run the regression, I get this error:TypeError: float() argument must be a string or a number, not 'Timestamp'Thanks again!

123

Kiran 29 May, 2019

Great article, couple of questions (I'm a beginner with DL, dabbled with SkLearn before) ..1) How do i reproduce the LSTM model on my local Windows machine (or in a Google cloud account for AI, if that's easier)? Do i install Conda on Windows or some other stack? [The Github/fastai clone is supported only on Linux machines]2) Could you post the daily prediction and actual prices through the validation period? I'm trying to assess the Win-rate (i.e. accuracy of predicted direction) and the Net Profits and Net Losses from a daily investing strategy that invested Long/Short daily based on the prediction. It's hard to tell from the chart - typically, unless there's 60% win-rate and Profit Factor (i.e. Net Profit/Net Loss) of 2x, it's not tradable.

123

Jonathan Lim 30 May, 2019

Hi Aishwarya,My name is Jonathan Lim, and I have some experience in data science and ML. First of all, thank you so much for this post! I have been trying to figure how to predict the stock market for a while now, and your post ties up the knot between ML and stock market prediction really well!Thus, I would love to learn more about predicting the stock market using ML from you and to hear more about your experiences working in the data science and ML field! :)I have attached my LinkedIn profile along with this comment, which I hope to connect with you. Thank you, and I hope to hear from you soon!Sincerely,Jonathan Lim

123

Faheem Parach 14 Jun, 2019

Plz would anyone send me complete code without errors. ? i am facing a lot of issues.

123

sachin 17 Jun, 2019

Thanks Aishwarya,i am getting below error:from fastai.structured import add_datepart---------------------------------------------------------------------------ModuleNotFoundError Traceback (most recent call last)in----> 1 from fastai.structured import add_datepartModuleNotFoundError: No module named 'fastai'i tried pip install fastaistill no luckcan you please help me with this

123

Raj 17 Jun, 2019

Hi,While predicting you use last 60 actual values from train and predict the first value of Pred[0].Then you use the last 59 actual of train and 1 actual value from valid and get Pred[1] and so on...Thus not using the predicted values to further predict values but rather using all actual values and making separate 1 day predictions. Is my understanding right?

123

Jay 17 Jun, 2019

Hi Aishwarya SinghIts really nice article,can you please share me latest datasetThanks

123

Tamilselvi 19 Jun, 2019

Hi aish,Can u share ideas little deeper on how to predict future sales for two more days.it wil be helpful

123

Sunil Kelker 19 Jun, 2019

How does one add a feature to be evaluated for predictions. For example Volume, to the LSTM.All help appreciated.

123

O P MISHRA 19 Jan, 2022

Hi Aishwarya,I am writing you on a thread that has no comments for last 20 months. I read the article and found it interesting. I have a few questions as well as a strategy that I want to automate. Seeing your research and interest in the topic, maybe we can collaborate.Plz revert if you would want to discuss it further.

123

Rajkumar Telukuntla 19 Jan, 2022

Hi! Thanks a lot for this great article. I'm getting the below errorKeyError Traceback (most recent call last)~\AppData\Local\Temp/ipykernel_23052/3766128780.py in4344 #predicting 246 values, using past 60 from the train data---> 45 inputs = new_data[len(new_data) - len(valid) - 60:].values46 inputs = inputs.reshape(-1,1)47 inputs = scaler.transform(inputs)~\anaconda3\lib\site-packages\pandas\core\frame.py in __getitem__(self, key)34283429 # Do we have a slicer (on rows)?-> 3430 indexer = convert_to_index_sliceable(self, key)3431 if indexer is not None:3432 if isinstance(indexer, np.ndarray):

123

Nahush 09 Feb, 2022

Can someone suggest a code to predict values for the next 30 days, whose values are not present in the dataset?

123

Dasheek Naidu 20 Feb, 2022

Hi Aishwarya, please may you share the full working code with me. Really appreciate it!

123

TS 21 Mar, 2022

Hi, please help - how to provide input data (invest 10000) for 10 days using the LSTM model and get the predicted value. Can't find any place to provide those inputs.

123

Jain Patel 27 Oct, 2022

Hey my accuracy is in negative for all model. Can you please guide me?

123

Related Courses

0 Hrs72 Lessons

4.84

A Comprehensive Learning Path for Deep Learning in 2023

Free

Learning PathDeep LearningNeural Networks

Enroll now

0 Hrs72 Lessons

4.94

A Comprehensive Learning Path for Deep Learning in 2020

Free

Learning PathDeep LearningNeural Networks

Enroll now

0 Hrs76 Lessons

4.93

A Comprehensive Learning Path for Deep Learning in 2019

Free

Learning PathDeep LearningNeural Networks

Enroll now

Write for usWrite, captivate, and earn accolades and rewards for your work

Rahul Shah27

Sion Chakrabarti16

CHIRAG GOYAL87

Barney Darlington5

Suvojit Hore9

Arnab Mondal15

Prateek Majumder68

Welcome to India's Largest Data Science Community

google-iconContinue with Google

I accept the Terms and Conditions

Receive Updates on Whatsapp

Welcome Back

I accept the Terms and Conditions

Receive Updates on Whatsapp

I accept the Terms and Conditions

Receive Updates on Whatsapp

A verification link has been sent to your email id

If you have not recieved the link please gotoSign Up page again

Please enter the OTP that is sent to your registered email id

Please enter the OTP that is sent to your email id

Please enter your registered email id

This email id is not registered with us. Please enter your registered email id.

Don't have an account yet?Register here

Please enter the OTP that is sent your registered email id

Please create the new password here

We use cookies on Analytics Vidhya websites to deliver our services, analyze web traffic, and improve your experience on the site. By using Analytics Vidhya, you agree to our Privacy Policy and Terms of Use.Accept

Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.

Necessary

Always Enabled

Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.

Non-necessary

Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.

SAVE & ACCEPT

' );//setTimeout(function() {$(".interview-answers").before( '

' ) }, 5000);//setTimeout(function() { $(".interview-answers").hide(); }, 5000);//$(".interview-answers").hide();}// Bokmark Article$(document).on("click",".bookmark",function(){ var bookmark_post_id = $(this).data('id'); var bookmark_post_title = $(this).data('title'); var bookmark_post_link = $(this).data('link');if (av_is_signed_in)bookmark_post(bookmark_post_id,bookmark_post_link,bookmark_post_title,this); elsestartAuthProcess(BLOG_BOOKMARK_BTN_ID, window.location.href); // Call Simplified Reg//window.location = id_login_url;});$('textarea[name=comment]').click(function(){ if (av_is_signed_in) console.log("login"); else startAuthProcess(BLOG_BOOKMARK_BTN_ID, window.location.href); // Call Simplified Reg});$(".download-pdf").click(function(){if (av_is_signed_in){ var post_id = $(this).data('id');datalayer_download_pdf_click();window.location = "https://www.analyticsvidhya.com/back-channel/download-pdf.php?pid="+post_id+"&next="+$(location).attr('href');}elsestartAuthProcess(BLOG_PDF_DOWNLOAD_BTN_ID, window.location.href); // Call Simplified Reg //window.location = id_login_url;});function bookmark_post(post_id,post_link,post_title,object){event.preventDefault();datalayer_bookmark_click();var bookmark_data = {resource_id:post_id, resource_type:"article",resource_permalink:post_link,resource_title:post_title,resource_medium: "blog"}; if (is_bookmarked==false) { $.ajax({type:"POST",//dataType:"json",url:"https://id.analyticsvidhya.com/api/v1/resources/external_save",data:JSON.stringify(bookmark_data), contentType: 'application/json',success: function(data) {data =JSON.stringify(data); //$("#bookmark-article").addClass("fas").attr("title","Remove Bookmark"); $(object).addClass("bookmarked").attr("title","Remove Bookmark"); is_bookmarked=true;}, error: function(data) { data = JSON.stringify(data); //$("#bookmark-article").removeClass("fas").attr("title","Failed to Bookmark Article"); $(object).removeClass("bookmarked").attr("title","Failed to Bookmark Article"); } });} // end ifelse if (is_bookmarked==true){ $.ajax({type:"DELETE",dataType:"json",url:"https://id.analyticsvidhya.com/api/v1/resources/external_save",data:JSON.stringify(bookmark_data), contentType: 'application/json',success: function(data) {data =JSON.stringify(data);//$("#bookmark-article").removeClass("fas").attr("title","Bookmark this story to read later");$(object).removeClass("bookmarked").attr("title","Bookmark this story to read later"); is_bookmarked=false;}, error: function(data) { data = JSON.stringify(data);// $("#bookmark-article").attr("title","Failed to Remove Bookmark"); $(object).attr("title","Failed to Remove Bookmark");} });} //end elseif}function initCodingWindow() {$('.coding-window iframe').each(function() {var src = $(this).data('src');if (!src) {return;}this.src = src;})}$('#loginModal').on('show.bs.modal', function (event) {if (av_is_signed_in)$('#loginModal').modal("hide");});// $('#loginModal').on('shown.bs.modal', function (event) {// if (av_is_signed_in)// $('#loginModal').modal("hide");// });function get_cookie(cookiename) { var cookielist = document.cookie.split(";"); for(var i = 0; i < cookielist.length; i++) { var cookiedata = cookielist[i].split("="); if(cookiename == cookiedata[0].trim()) { return decodeURIComponent(cookiedata[1]); } } return false;}// Add Utm Campaign as Page Url$(document).ready(function(){ $('.utm_image_link').each(function() { var href = $(this).attr('href'); if (href) { var url = new URL(href); url.searchParams.delete("utm_campaign"); url.searchParams.set("utm_campaign", "https://www.analyticsvidhya.com/blog/2018/10/predicting-stock-price-machine-learningnd-deep-learning-techniques-python/"); $(this).attr('href', url.href); }});// Old Login$(".header-sign-in").click(()=>{//window.location = id_login_url; });$(".simplified-login").click(()=>{ startAuthProcess(BLOG_SIGN_IN_BTN_ID, window.location.href); }); $(".simplified-login-join-now").click(()=>{ startAuthProcess(BLOG_SIGN_IN_BTN_ID, window.location.href); });// Logout from ID $("#logout-from-id").click(()=>{ logout(); }); // update claps $('#post-likes').click(function(){ var post_id = $(this).data('id'); if(av_is_signed_in){ $.ajax({ url: "https://www.analyticsvidhya.com/wp-content/themes/analytics-vidhya/news-letter-submit-form.php", method: "POST", data: {post_id:post_id,type:"updatelikes"}, success: function(result){ // $('input[name="email"]').val(); if(result !='' && result !='Please enter Post ID'){ result = JSON.parse(result); if(result.data.updateLikes.success){ $('.clap').hide(); $('.claped').show(); $('#likes-count').html(result.data.updateLikes.hitCount); } else console.log("post likes failed"); }else{ console.log(result); } } }); }else{ startAuthProcess(BLOG_SIGN_IN_BTN_ID, window.location.href); // Call Simplified Reg } }); // get claps $.ajax({ url: "https://www.analyticsvidhya.com/wp-content/themes/analytics-vidhya/news-letter-submit-form.php", method: "POST", data: {post_id:47846,type:"getlikes"}, success: function(result){ //console.log(result); if(result !='' && result != 0 && result !='Please enter Post ID'){ $('#likes-count').html(result); } } });});

'; $('.yoast-table-of-contents').html(acrd); //add new design of FAQs var faq_question = ''; var faq_answer = ''; var id = ''; var new_faq_content = '';$('.schema-faq-section').each(function(index,element){ // if(!$(element).hasClass('collapse')){ id = $(element).attr('id'); faq_question = $(element).find(' .schema-faq-question strong').html(); faq_answer = $(element).find(' .schema-faq-answer').html(); faq_question = (typeof(faq_question) === "undefined"?$(element).find(' .schema-faq-question').html():faq_question); new_faq_content += '

'+faq_answer+'

'; // }});//console.log(new_faq_content);if(new_faq_content != ''){ $('.wp-block-yoast-faq-block').html(''); $('#h-frequently-asked-questions').addClass('d-none'); $('#accordionFaq').html(new_faq_content); $('#faq').removeClass('d-none');} var myOffcanvas = document.getElementById('offcanvasScrolling') if(myOffcanvas !== null){ myOffcanvas.addEventListener('hidden.bs.offcanvas', function () { if($('#accordionReadingList .accordion-item:first-child h2 button.accordion-button').hasClass('collapsed')){ $('#accordionReadingList .accordion-item:first-child h2 button.accordion-button').removeClass('collapsed') $('#accordionReadingList .accordion-item:first-child .accordion-collapse').addClass('show') } if($("#accordionReadingList .accordion-item:not(:first-child) h2 button.accordion-button").hasClass('collapsed')){ $("#accordionReadingList .accordion-item:not(:first-child) h2 button.accordion-button").addClass('collapsed') $("#accordionReadingList .accordion-item:not(:first-child) .accordion-collapse").removeClass('show') } }) }});

Stock Prices Prediction Using Machine Learning and Deep Learning (2024)

FAQs

Can deep learning predict stock prices? ›

Training the Model

Training a deep learning model for stock price prediction involves feeding historical price sequences into the LSTM network and using backpropagation through time (BPTT) to optimize the model's parameters.

Explore More ›

Which machine learning model is best for stock price prediction? ›

However, LSTM model is more appropriate in short-term stock price forecasting in the field of machine learning.

Tell Me More ›

How to use AI to predict stock price? ›

AI stock prediction software works by evaluating bulk financial data to help you have the most important data insights for stock selection. Machine learning algorithms, NLP, algorithmic strategies, and other such components help create a fine-tuned stock prediction app.

Get More Info Here ›

Can AI really predict stock market? ›

"We found that these AI models significantly outperform traditional methods. The machine learning models can predict stock returns with remarkable accuracy, achieving an average monthly return of up to 2.71% compared to about 1% for traditional methods," adds Professor Azevedo.

Explore More ›

How accurate is LSTM stock price prediction? ›

The main objective is to forecast the current market trends and could predict the stock prices accurately. We use LSTM recurrent neural networks to predict the stock prices accurately. The results show that prediction accuracy is over 93%.

Discover More Details ›

What is the most accurate stock predictor AI? ›

Danelfin – Danelfin is an AI stock predictor that covers thousands of companies from the NYSE and NASDAQ. It also analyzes stocks from the European markets. Danelfin generated growth of 158% between January 2017 and December 2022. In the same period, the S&P 500 grew by 70%.

What are the disadvantages of stock market prediction using machine learning? ›

What are the Challenges and Limitations of Stock Price Prediction Using Machine Learning?

Data Volatility. Stock prices are influenced by a multitude of factors, including news, geopolitical events, and market sentiment. ...
Nonlinearity. ...
Limited Historical Data. ...
Overfitting. ...
Data Quality and Bias.

Sep 28, 2023

Discover More ›

Which machine learning algorithm is best for trading? ›

Below are the most used Machine Learning algorithms for quantitative trading:

Linear Regression.
Logistic Regression.
Random Forests (RM)
Support Vector Machine (SVM)
k-Nearest Neighbor (KNN)
Classification and Regression Tree (CART)
Deep Learning algorithms.

Which AI is best for the stock market? ›

HoopAI, which uses powerful AI technology, is aimed to cater to individual retail investors in the stock market by providing individualized trading ideas based on tastes and needs.

Show Me More ›

Can ChatGPT 4 predict stocks? ›

ChatGPT is a comprehensive artificial intelligence language model that has been trained to engage in human-like conversations, generate texts, and provide users with answers to their questions. Moreover, it has recently been able to correctly predict stock market changes.

Find Out More ›

Why can't AI predict the stock market? ›

If there are significant changes in market conditions or new factors influencing the market, the algorithm may struggle to adapt and accurately predict future behavior. Furthermore, AI algorithms can also be susceptible to manipulations and biases, as they are only as unbiased as the data they are trained on.

Can CNN be used for stock prediction? ›

Because the stock data can be seen as a large 2D matrix, [3] has used ANN model to make prediction and gain a satisfied result, both of which have proved that CNN also can be used to do the same thing.

See Details ›

What is the most accurate stock predictor? ›

Zacks Ultimate has proven itself as one of the most accurate stock predictors for more than three decades. Incepted in 1988, this established service has produced phenomenal returns for its members. In fact, since 1998, Zacks Ultimate has generated average annualized returns of 24.3%.

Know More ›

How CNN can predict stock price? ›

Recently, CNN is now used in Natural Language Processing (NLP) based applications, so by identifying the features from stock data and converting them into tensors, we can obtain the features and then send it to LSTM neural network to find the patterns and thereby predicting the stock market for given period of time.

Tell Me More ›

Can LSTM be used for stock prediction? ›

It is often used in analyzing time series data, such as stock market prices. Time series data refers to the sequence of past stock prices and other financial data, and LSTM can use this data to identify patterns and trends that can be used to predict future stock prices.

Read The Full Story ›

Stock Prices Prediction Using Machine Learning and Deep Learning (2024)

Introduction

Table of contents

Understanding the Problem Statement

Moving Average

Introduction

Implementation

Inference

Linear Regression

Introduction

Implementation

Results

Inference

k-Nearest Neighbours

Introduction

Implementation

Results

Inference

Auto ARIMA

Introduction

Implementation

Results

Inference

Prophet

Introduction

Implementation

Results

Inference

Long Short Term Memory (LSTM)

Introduction

Implementation

Results

Inference

Conclusion

Frequently Asked Questions

Frequently Asked Questions

Responses From Readers

Related Courses

Free

Free

Free

Welcome to India's Largest Data Science Community

Welcome Back

Please enter the OTP that is sent to your registered email id

Please enter the OTP that is sent to your email id

Please enter your registered email id

Don't have an account yet?Register here

Please enter the OTP that is sent your registered email id

Please create the new password here

Privacy Overview

FAQs

Can deep learning predict stock prices? ›

Why can't AI predict the stock market? ›

References